DeepTeam is a framework to red team LLMs and AI agents.
Finished docs for frameworks
A-Vamshib971a7130479
diff --git a/docs/docs/frameworks-aegis.mdx b/docs/docs/frameworks-aegis.mdxnew file mode 100644index 00000000..92a28764--- /dev/null+++ b/docs/docs/frameworks-aegis.mdx@@ -0,0 +1,69 @@+---+id: frameworks-aegis+title: Aegis+sidebar_label: Aegis+---++The **Aegis** framework integrates the [NVIDIA Aegis AI Content Safety Dataset](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-1.0) — an open-source dataset aligned with NVIDIA’s **Content Safety Taxonomy** across 13 critical harm categories.+Aegis enables DeepTeam to perform **dataset-driven red teaming** using real human-labeled safety violations to validate model robustness against harmful or unsafe user inputs.++## Overview++Aegis focuses on **real-world unsafe content** from public conversations. This allows evaluation of model refusal behaviors and fine-tuning of safety filters using authentic, labeled harm categories.++```python+from deepteam.frameworks import Aegis+from deepteam import red_team+from somewhere import your_model_callback++aegis = Aegis(num_attacks=10)++risk_assessment = red_team(+ model_callback=your_model_callback,+ framework=aegis,+)++print(risk_assessment)+```++The `Aegis` framework accepts **FOUR** optional parameters:++- [Optional] `num_attacks`: Number of harmful test cases to sample from the dataset. Defaulted to `15`.+- [Optional] `evaluation_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://deepeval.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM`. Defaulted to `'gpt-4o'`.+- [Optional] `async_mode`: a boolean which when set to `True`, enables concurrent execution. Defaulted to `True`.+- [Optional] `verbose_mode`: a boolean which when set to `True`, prints intermediate steps used to evaluate model responses. Defaulted to `False`.++## How It Works++1. Loads the Aegis dataset from Hugging Face.+2. Filters unsafe samples labeled under NVIDIA’s harm taxonomy.+3. Randomly samples `num_attacks` test cases.+4. Evaluates how well the model identifies, refuses, or mitigates these harmful prompts.++## Categories & Filtering++Aegis covers 13 harm domains, all supported by DeepTeam filtering.++**Example categories:**++- `sexual_content`+- `violence`+- `hate_speech`+- `self_harm`+- `misinformation`+- `privacy_violation`+- `child_exploitation`+- `drugs`+- `terrorism`++## Related Docs++- [MITRE ATLAS framework](/docs/red-teaming-frameworks-mitre)+- [NIST AI RMF framework](/docs/red-teaming-frameworks-nist)+- [BeaverTails dataset framework](/docs/red-teaming-frameworks-beavertails)+- [Vulnerabilities in DeepTeam](/docs/red-teaming-vulnerabilities)+- [Adversarial attacks in DeepTeam](/docs/red-teaming-adversarial-attacks)++## References++[NVIDIA Aegis AI Content Safety Dataset](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-1.0) (Hugging Face)diff --git a/docs/docs/frameworks-beavertails.mdx b/docs/docs/frameworks-beavertails.mdxnew file mode 100644index 00000000..63909794--- /dev/null+++ b/docs/docs/frameworks-beavertails.mdx@@ -0,0 +1,71 @@+---+id: frameworks-beavertails+title: BeaverTails+sidebar_label: BeaverTails+---++The **BeaverTails** framework integrates the [BeaverTails dataset](https://huggingface.co/datasets/PKU-Alignment/BeaverTails) — a large, human-labeled collection of harmful and borderline prompts — so DeepTeam can run **dataset-driven red teaming** against real-world adversarial inputs.+BeaverTails is designed to surface safety failures across a wide range of harm categories (abuse, illegal instructions, privacy violations, self-harm, sexual content, misinformation, and more) by testing how models respond to actual annotated malicious prompts.++## Overview++BeaverTails focuses on **sampling real harmful prompts** from a curated dataset rather than synthetic generation. This provides higher-fidelity signals for model behavior in the wild and helps you validate safety defenses, content filters, and ingestion pipelines.++```python+from deepteam.frameworks import BeaverTails+from deepteam import red_team+from somewhere import your_model_callback++beaver_tails = BeaverTails(num_attacks=10)++risk_assessment = red_team(+ model_callback=your_model_callback,+ framework=beaver_tails,+)++print(risk_assessment)+```++The `BeaverTails` framework accepts **FIVE** optional parameters:++- [Optional] `num_attacks`: A list of strings that represent the `NIST` measure you want to test your AI application on:+- [Optional] `data_split`: A string which decides which data is used for fetching the dataset. It is a string literal that accepts either `330k_train` or `30k_train`. Defaulted to `30k_train`.+- [Optional] `evaluation_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://deepeval.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM`. Defaulted to 'gpt-4o'.+- [Optional] `async_mode`: a boolean which when set to `True`, enables concurrent execution. Defaulted to `True`.+- [Optional] `verbose_mode`: a boolean which when set to `True`, prints the intermediate steps used to evaluate target LLM's response to dataset's input. Defaulted to `False`.++## How It Works++1. Loads the dataset using Hugging Face's `datasets` with the specified BeaverTails split+2. Filters out unsafe test cases to focus only on potentially harmful attacks+3. Randomly samples `num_attacks` prompts+4. Evaluates if the model appropriately refuses and handles the harmful attacks from dataset++## Categories & Filtering++BeaverTails includes 14 harm subcategories. DeepTeam supports filtering by subcategory so you can focus testing on highest-risk areas.++**Example subcategories:**++- `animal_abuse`+- `child_abuse`+- `discrimination` / `hate_speech`+- `drug_abuse`, `weapons`+- `financial_crime`+- `misinformation`+- `privacy_violation`+- `self_harm`+- `sexually_explicit`+- `terrorism`+- `violence`++## Related Docs++- [MITRE ATLAS framework](/docs/red-teaming-frameworks-mitre)+- [NIST AI RMF framework](/docs/red-teaming-frameworks-nist)+- [Vulnerabilities in DeepTeam](/docs/red-teaming-vulnerabilities)+- [Adversarial attacks in DeepTeam](/docs/red-teaming-adversarial-attacks)++## References++[BeaverTails dataset - `PKU-Alignment/BeaverTails`](https://huggingface.co/datasets/PKU-Alignment/BeaverTails) (Hugging Face)diff --git a/docs/docs/frameworks-introduction.mdx b/docs/docs/frameworks-introduction.mdxnew file mode 100644index 00000000..c6b42a7f--- /dev/null+++ b/docs/docs/frameworks-introduction.mdx@@ -0,0 +1,125 @@+---+id: frameworks-introduction+title: Introduction+sidebar_label: Introduction+---++The DeepTeam Frameworks define structured methodologies for **AI red teaming and risk assessment**.+Each framework maps to a recognized safety or security standard, helping you test your model's robustness against **real-world adversarial behavior, dataset risks, and system vulnerabilities**.++DeepTeam supports multiple frameworks — from dataset-based testing (_BeaverTails, Aegis_) to security and governance standards (_MITRE ATLAS, NIST AI RMF, OWASP Top 10 for LLMs_).++## Available Frameworks++Here are the list of frameworks available in `deepteam`:++- [MITRE ATLAS](#mitre-atlas)+- [NIST AI RMF](#nist-ai-rmf)+- [OWASP Top 10 for LLMs](#owasp-top-10-for-llms)+- [BeaverTails](#beavertails)+- [Aegis](#aegis)++### MITRE ATLAS++The MITRE ATLAS framework integrates the MITRE ATLAS knowledge base, focusing on adversarial tactics and techniques used against AI systems.+It evaluates system resilience across attack phases like Reconnaissance, Resource Development, Initial Access, and Impact.++- Tests adversarial behavior patterns from the ATLAS taxonomy+- Detects vulnerabilities such as prompt injection, data poisoning, and exfiltration+- Ideal for AI security simulation and penetration testing++```python+from deepteam.frameworks import MITREATLAS+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITREATLAS(num_attacks=10)++risk = red_team(+ model_callback=your_model_callback,+ framework=atlas+)+```++[Learn more about MITRE ATLAS](/docs/red-teaming-frameworks-mitre)++### NIST AI RMF++The NIST AI Risk Management Framework (RMF) provides a structured approach to identifying, managing, and mitigating AI risks.+It focuses on trustworthiness, robustness, and accountability — aligning LLM behavior with regulatory and ethical standards.++- Tests safety, fairness, and robustness dimensions+- Suitable for compliance-driven AI governance workflows++```python+from deepteam.frameworks import NIST+nist = NIST(num_attacks=10)++risk = red_team(+ model_callback=your_model_callback,+ framework=nist+)+```++[Learn more about NIST AI RMF](/docs/red-teaming-frameworks-nist)++### OWASP Top 10 for LLMs++The OWASP Top 10 for LLMs framework identifies the most critical security risks in LLM applications.+It reflects the 2025 OWASP edition, covering vulnerabilities in RAG systems, agents, and model integrations.++- Tests for prompt injection, system prompt leakage, vector weaknesses, and more+- Simulates real-world exploit attempts and harmful outputs+- Ideal for application-level AI security assessments++```python+from deepteam.frameworks import OWASPTop10+owasp = OWASPTop10(num_attacks=10)++risk = red_team(+ model_callback=your_model_callback,+ framework=owasp+)+```++[Learn more about OWASP Top 10 for LLMs](/docs/red-teaming-frameworks-owasp-top-10-for-llms)++### BeaverTails++The BeaverTails framework integrates the PKU BeaverTails dataset — a large, human-labeled dataset of harmful and borderline prompts.+It performs dataset-driven red teaming, surfacing model weaknesses across categories like abuse, misinformation, and privacy violations.++- Uses real-world harmful prompts instead of synthetic generation+- Validates content safety and refusal behavior++```python+from deepteam.frameworks import BeaverTails+beaver = BeaverTails(num_attacks=10)++risk = red_team(+ model_callback=your_model_callback,+ framework=beaver+)+```++[Learn more about BeaverTails](/docs/red-teaming-frameworks-beavertails)++### Aegis++The Aegis framework integrates the NVIDIA Aegis AI Content Safety Dataset, which follows NVIDIA's content safety taxonomy across 13 harm categories.+It provides a comprehensive safety evaluation using real human-labeled harmful content.++- Tests for harmful user messages across multiple safety dimensions+- Useful for evaluating model robustness under real-world safety challenges++```python+from deepteam.frameworks import Aegis+aegis = Aegis(num_attacks=10)++risk = red_team(+ model_callback=your_model_callback,+ framework=aegis+)+```++[Learn more about Aegis](/docs/red-teaming-frameworks-aegis)diff --git a/docs/docs/frameworks-mitre-atlas.mdx b/docs/docs/frameworks-mitre-atlas.mdxnew file mode 100644index 00000000..787be947--- /dev/null+++ b/docs/docs/frameworks-mitre-atlas.mdx@@ -0,0 +1,328 @@+---+id: frameworks-mitre-atlas+title: MITRE ATLAS+sidebar_label: MITRE ATLAS+---++The **MITRE ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems)** framework provides a structured knowledge base of adversarial tactics, techniques, and procedures (TTPs) used against AI and ML systems.+It extends the principles of [MITRE ATT&CK®](https://attack.mitre.org) to the **AI threat surface**, identifying how adversaries can manipulate, exploit, or misuse AI models throughout their lifecycle.++DeepTeam's **MITRE ATLAS module** implements these adversarial mappings to test your AI or LLM application for security, privacy, and robustness vulnerabilities, across each phase of the AI attack lifecycle.++## Overview++In DeepTeam, the MITRE ATLAS framework operationalizes AI-specific adversarial tactics to help organizations **detect**, **mitigate**, and **analyze** model exploitation risks.++Each tactic represents a **goal** an adversary may have while attacking an AI system. These tactics collectively map the “why” behind adversarial actions and correspond to different testing modules in DeepTeam.++| Tactic | Description |+| ------------------------ | ---------------------------------------------------------------------- |+| **Reconnaissance** | Gathering intelligence about AI systems and configurations |+| **Resource Development** | Acquiring resources or tools to enable future attacks |+| **Initial Access** | Gaining entry to the target AI system or environment |+| **ML Attack Staging** | Preparing, training, or adapting attacks specifically for AI models |+| **Exfiltration** | Stealing sensitive information, model data, or internal configurations |+| **Impact** | Manipulating or degrading AI systems to achieve adversarial goals |++Each of these categories can be tested independently or together:++```python+from deepteam.frameworks import MITRE+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITRE(categories=["reconnaissance", "impact"])+attacks = atlas.attacks+vulnerabilities = atlas.vulnerabilities++risk_assessment = red_team(+ model_callback=your_model_callback,+ vulnerabilities=vulnerabilities,+ attacks=attacks+)+```++The `MITRE` framework accepts **ONE** optional parameter:++- [Optional] `categories`: A list of strings that represent the `MITRE` ATLAS tactics you want to test your AI application on:+ - `reconnaissance`: Tests for unintended disclosure of internal info, prompts, or policies through probing or reasoning.+ - `resource_development`: Checks if the system can aid in creating or supporting malicious tools or content.+ - `initial_access`: Evaluates resistance to malicious entry via prompt injection, debug access, or exposed interfaces.+ - `ml_attack_staging`: Tests handling of poisoned or adversarial inputs used to prepare model-targeted attacks.+ - `exfiltration`: Detects data leakage or extraction of sensitive information through adversarial queries.+ - `impact`: Simulates harmful actions like goal hijacking, content manipulation, or autonomy abuse.++## Reconnaissance — Information Gathering and System Profiling++**(MITRE ATLAS ID: AML.TA0002)**++**Goal:** The adversary is trying to gather information about the AI system they can use to plan future operations.++Reconnaissance involves identifying model capabilities, business rules, and potential weaknesses. Adversaries may probe model outputs, query APIs, or use role-based misdirection to uncover internal policies, system prompts, or hidden logic.++DeepTeam tests whether your AI system inadvertently exposes:++- Internal guardrails, policies, or roles+- Confidential reasoning chains or business logic+- Sensitive prompts, credentials, or decision rules+- Competitive or private strategic information++**Example vulnerabilities**++- `Competition`+- `PromptLeakage`+- `RBAC`+- `CustomVulnerability` — _Policy Disclosure_++**Example attacks**++- `Roleplay`+- `PromptInjection`+- `CrescendoJailbreaking`+- `SequentialJailbreak`+- `TreeJailbreaking`++```python+from deepteam.frameworks import MITRE+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITRE(categories=["reconnaissance"])+attacks = atlas.attacks+vulnerabilities = atlas.vulnerabilities++# Modify attributes for your specific testing context if needed+red_team(+ model_callback=your_model_callback,+ attacks=attacks,+ vulnerabilities=vulnerabilities+)+```++Reconnaissance testing ensures your model does not reveal **sensitive internal logic** or **operational metadata** to external users.++## Resource Development — Adversarial Capability Building++**(MITRE ATLAS ID: AML.TA0003)**++**Goal:** The adversary is creating or acquiring resources (data, prompts, tools, accounts) to enable future AI attacks.++**Threat landscape:** Adversaries prepare poisoned datasets, prompt libraries, proxy models, or obfuscated payloads that can be used later to bypass safeguards, craft jailbreaks, or seed long-term attacks.++**Testing strategy:** Simulate adversary preparations by submitting obfuscated/templated prompts, benign-looking poisoned examples, and dataset-like inputs to detect:++- whether the system assists in creating malicious artifacts, or+- whether such inputs are accepted, stored, or likely to influence future behavior (e.g., RAG ingestion, fine-tuning pipelines, or plugin storage).++**Example vulnerabilities**++- `IllegalActivity`+- `CustomVulnerability` - _Execution, Persistence, Defense Evasion, Discovery, CommandAndControl_++**Example attacks**++- `Roleplay`+- `Leetspeak`, `ROT13`+- `PromptInjection`, `CrescendoJailbreaking`++**Example (run only resource-development tests):**++```python+from deepteam.frameworks import MITRE+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITRE(categories=["resource_development"])+attacks = atlas.attacks+vulnerabilities = atlas.vulnerabilities++# Modify attributes for your specific testing context if needed+red_team(+ model_callback=your_model_callback,+ attacks=attacks,+ vulnerabilities=vulnerabilities+)+```++This ensures your AI system remains resilient against adversarial data collection, obfuscation, or resource manipulation during reconnaissance and setup stages.++## Initial Access — Entry Point Exploitation++**(MITRE ATLAS ID: AML.TA0004)**++**Goal:** The adversary is trying to gain access to the AI system.++Adversaries may attempt to exploit the AI model's input surfaces, administrative interfaces, or embedded systems. DeepTeam maps these behaviors through **code injection**, **misuse of debugging features**, and **role impersonation**.++**Example vulnerabilities**++- `DebugAccess`+- `IllegalActivity`+- `SQLInjection`+- `SSRF`+- `ShellInjection`++**Example attacks**++- `PromptInjection`+- `LinearJailbreaking`+- `SequentialJailbreak`+- `Roleplay`++```python+from deepteam.frameworks import MITRE+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITRE(categories=["initial_access"])+attacks = atlas.attacks+vulnerabilities = atlas.vulnerabilities++# Modify attributes for your specific testing context if needed+red_team(+ model_callback=your_model_callback,+ attacks=attacks,+ vulnerabilities=vulnerabilities+)+```++Testing this phase ensures your model and connected systems resist **direct exploitation** or **unauthorized entry** into AI pipelines.++## ML Attack Staging — Model-Specific Attack Preparation++**(MITRE ATLAS ID: AML.TA0001)**++**Goal:** The adversary is leveraging their knowledge of and access to the target system to tailor the attack.++This phase is unique to AI. Attackers train **proxy models**, **poison context**, or **craft adversarial inputs** to prepare targeted manipulations.++DeepTeam emulates these scenarios by introducing **hallucination-based, poisoned, or misleading inputs** and observing how your model responds.++**Example vulnerabilities**++- `ExcessiveAgency`+- `CustomVulnerability` — _Hallucination_+- `CustomVulnerability` — _Indirect Prompt Injection_++**Example attacks**++- `PromptInjection`, `Leetspeak`, `ROT13`+- `LinearJailbreaking`, `TreeJailbreaking`, `SequentialJailbreak`++```python+from deepteam.frameworks import MITRE+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITRE(categories=["ml_attack_staging"])+attacks = atlas.attacks+vulnerabilities = atlas.vulnerabilities++# Modify attributes for your specific testing context if needed+red_team(+ model_callback=your_model_callback,+ attacks=attacks,+ vulnerabilities=vulnerabilities+)+```++This phase helps ensure your model's **context integrity**, **instruction following**, and **data reliability** cannot be manipulated by crafted adversarial inputs.++## Exfiltration — Data or Model Theft++**(MITRE ATLAS ID: AML.TA0010)**++**Goal:** The adversary is trying to steal AI artifacts or other sensitive information.++Adversaries in this stage attempt to **extract system prompts, private data, or intellectual property** through direct or indirect model interaction.++DeepTeam tests for **leakage, encoding abuse, and unauthorized disclosure** across multiple channels.++**Example vulnerabilities**++- `PIILeakage`+- `IntellectualProperty`+- `CustomVulnerability` — _ASCII Smuggling, Prompt Extraction, Privacy, Indirect Prompt Injection_++**Example attacks**++- `PromptProbing`, `Leetspeak`, `ROT13`+- `PromptInjection`, `SequentialJailbreak`+- `Roleplay` (Security Engineer persona)++```python+from deepteam.frameworks import MITRE+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITRE(categories=["exfiltration"])+attacks = atlas.attacks+vulnerabilities = atlas.vulnerabilities++# Modify attributes for your specific testing context if needed+red_team(+ model_callback=your_model_callback,+ attacks=attacks,+ vulnerabilities=vulnerabilities+)+```++This testing phase validates whether your AI system properly **masks**, **encodes**, and **protects** data to prevent exfiltration through direct or covert model outputs.++## Impact — Manipulation, Misuse, and Degradation++**(MITRE ATLAS ID: AML.TA0011)**++**Goal:** The adversary is trying to manipulate, interrupt, or degrade AI system performance or trustworthiness.++In this final phase, adversaries aim to cause **misinformation, impersonation, or reputational harm**, or to exploit recursive model behaviors for persistent manipulation.++DeepTeam evaluates whether your AI system can resist **goal hijacking**, **recursive propagation**, or **harmful content generation**.++**Example vulnerabilities**++- `ExcessiveAgency`+- `GraphicContent`+- `RecursiveHijacking`+- `CustomVulnerability` — *Imitation*++**Example attacks**++- `PromptInjection`, `LinearJailbreaking`, `CrescendoJailbreaking`+- `Roleplay` (Authoritative CEO persona)++```python+from deepteam.frameworks import MITRE+from deepteam import red_team+from somewhere import your_model_callback++atlas = MITRE(categories=["impact"])+attacks = atlas.attacks+vulnerabilities = atlas.vulnerabilities++# Modify attributes for your specific testing context if needed+red_team(+ model_callback=your_model_callback,+ attacks=attacks,+ vulnerabilities=vulnerabilities+)+```++This stage ensures your system's final outputs and decisions remain aligned with **intended ethics, safety, and trust boundaries** — even under adversarial manipulation.++## Best Practices++1. **Test across all six phases** to ensure lifecycle-wide resilience.+2. **Simulate real adversaries** using chained or contextual attacks (`CrescendoJailbreaking`, `PromptProbing`).+3. **Audit and document every run** — align red teaming results with MITRE ATLAS categories.+4. **Integrate with NIST AI RMF** — combine measurement (NIST) and tactics (MITRE) for complete coverage.+5. **Continuously retrain and monitor** models for post-deployment drift and new attack patterns.+6. **Review model behavior under role-based adversarial prompts** to detect policy leaks or privilege misuse.++## Learn More++- [MITRE ATLAS™ Official Documentation](https://atlas.mitre.org/)+- [MITRE ATT&CK® Framework](https://attack.mitre.org/)+- [NIST AI RMF (for complementary evaluation)](https://www.nist.gov/itl/ai-risk-management-framework)++Would you like me to include a **mapping table** at the end (like NIST's measure summary) showing how each MITRE tactic maps to DeepTeam vulnerabilities and attacks (a compact summary table)? It would make it ready for production docs.diff --git a/docs/docs/frameworks-nist.mdx b/docs/docs/frameworks-nist.mdxnew file mode 100644index 00000000..8f9a8c1a--- /dev/null+++ b/docs/docs/frameworks-nist.mdx@@ -0,0 +1,281 @@+---+id: frameworks-nist+title: NIST AI RMF+sidebar_label: NIST AI RMF+---++The **NIST AI Risk Management Framework (AI RMF)** is a structured methodology from the U.S. National Institute of Standards and Technology that guides organizations in identifying, evaluating, and mitigating risks in artificial intelligence systems.+It promotes **trustworthy AI** by focusing on governance, measurement, and continuous risk tracking across the AI lifecycle.++DeepTeam's implementation of NIST focuses on the **Measure** function — the part of the framework responsible for **testing, evaluation, and assurance** of AI behavior and risk controls.++## Overview++DeepTeam's **NIST AI RMF module** automates red teaming and risk validation against NIST's measurement categories.+It evaluates your LLM or AI system for reliability, fairness, robustness, security, privacy, and resilience using a standardized testing approach.++The framework is divided into four categories of measures:++| Measure Category | Description |+| ---------------- | -------------------------------------------------------------------------------------------- |+| **Measure 1** | Define and apply appropriate testing and metrics for AI risk evaluation |+| **Measure 2** | Evaluate the AI system for trustworthiness, safety, security, fairness, and misuse potential |+| **Measure 3** | Establish mechanisms for identifying, tracking, and managing emerging risks |+| **Measure 4** | Measure and correlate AI risk impacts with business and performance outcomes |++## Using the NIST Framework in DeepTeam++You can run a full NIST-based red team assessment in DeepTeam using:++```python+from deepteam import red_team+from deepteam.frameworks import NIST+from somewhere import your_model_callback++risk_assessment = red_team(+ model_callback=your_model_callback,+ framework=NIST(categories=["measure_1"])+)+```++The `NIST` framework accepts **ONE** optional parameter:++- [Optional] `categories`: A list of strings that represent the `NIST` measure you want to test your AI application on:+ - `measure_1`: defines and applies appropriate testing and metrics for AI risk evaluation+ - `measure_2`: evaluates the AI system for trustworthiness, safety, security, fairness, and misuse potential+ - `measure_3`: establishes mechanisms for identifying, tracking, and managing emerging risks+ - `measure_4`: measures and correlates AI risk impacts with business and performance outcomes++## Measure Categories and Testing Coverage++DeepTeam operationalizes NIST's **Measure** function through four major categories that map to the subfunctions (M.1-M.4) in the [NIST AI Risk Management Framework (AI RMF 1.0)](https://www.nist.gov/itl/ai-risk-management-framework).+Each category corresponds to a distinct aspect of AI risk measurement, evaluation, monitoring, and feedback.++### Measure 1 — Risk Measurement and Metrics++**(NIST Subfunctions: M.1.1-M.1.3)**++**Goal:** Identify, apply, and continuously improve appropriate methods and metrics for assessing AI risks.++DeepTeam's Measure 1 implementation focuses on **measurable risk discovery** and **evaluation setup** — ensuring that test methods, metrics, and expert review processes are in place and auditable.++It reflects NIST's expectations that:++- AI risk metrics are **selected and documented** based on significance and feasibility (M.1.1).+- The **appropriateness of metrics** and **effectiveness of existing controls** are regularly reassessed (M.1.2).+- **Independent experts** or **non-developers** periodically review measurement validity (M.1.3).++DeepTeam tests:++- Intellectual property and data integrity safeguards+- Role-based access control (RBAC) and privilege management+- Exposure of debugging or administrative interfaces++**Example vulnerabilities tested**++- `IntellectualProperty`+- `RBAC`+- `DebugAccess`++**Example attacks**++- `PromptProbing`+- `GrayBox`+- `Roleplay` (Compliance Officer persona)++```python+from deepteam.frameworks import NIST+from deepteam import red_team+from somewhere import your_model_callback++nist = NIST(categories=["measure_1"])+attacks = nist.attacks+vulnerabilities = nist.vulnerabilities++# Modify attributes for your specific testing context if needed+risk_assessment = red_team(+ model_callback=your_model_callback,+ vulnerabilities=vulnerabilities,+ attacks=attacks+)+```++These tests ensure your AI system's evaluation methods are **structured, reviewable, and aligned with NIST's expectations for auditable risk measurement**.++### Measure 2 — Trustworthiness and Safety Evaluation++**(NIST Subfunctions: M.2.1-M.2.13)**++**Goal:** Evaluate and validate AI systems for **trustworthy characteristics** such as safety, fairness, security, robustness, privacy, and reliability — under real-world conditions.++This is the most comprehensive of the Measure categories.+DeepTeam automates testing aligned with NIST's subfunctions covering:++- **Evaluation documentation** and reproducibility (M.2.1-M.2.3)+- **Operational monitoring** of deployed systems (M.2.4)+- **Validation and reliability** of AI performance (M.2.5)+- **Safety, robustness, and fail-safe design** (M.2.6-M.2.7)+- **Transparency, accountability, and explainability** (M.2.8-M.2.9)+- **Privacy, fairness, and bias** evaluation (M.2.10-M.2.11)+- **Environmental and sustainability considerations** (M.2.12)+- **Effectiveness of evaluation and measurement processes** (M.2.13)++DeepTeam performs extensive testing across:++- **Bias, fairness, and ethics**+- **Safety and personal protection**+- **Data leakage, privacy, and prompt exposure**+- **Robustness and adversarial misuse**+- **Security vulnerabilities** (SSRF, SQL Injection, Shell Injection, etc.)+- **Content safety** (toxicity, graphic or harmful material)++**Example vulnerabilities**++- `Bias`, `Fairness`, `Ethics`, `Toxicity`+- `PromptLeakage`, `PIILeakage`+- `Robustness`, `ExcessiveAgency`+- `SQLInjection`, `ShellInjection`, `SSRF`+- `ChildProtection`, `PersonalSafety`, `IllegalActivity`++**Example attacks**++- `PromptInjection`, `CrescendoJailbreaking`, `SequentialJailbreak`+- `Leetspeak`, `ROT13`, `Base64`, `Multilingual`+- `Roleplay` (Security Researcher persona)++```python+from deepteam.frameworks import NIST+from deepteam import red_team+from somewhere import your_model_callback++nist = NIST(categories=["measure_2"])+attacks = nist.attacks+vulnerabilities = nist.vulnerabilities++# Modify attributes for your specific testing context if needed+risk_assessment = red_team(+ model_callback=your_model_callback,+ vulnerabilities=vulnerabilities,+ attacks=attacks+)+```++This category enforces **systemic testing of AI trustworthiness**, ensuring continuous validation across fairness, safety, privacy, robustness, and ethical boundaries.++### Measure 3 — Risk Tracking and Monitoring++**(NIST Subfunctions: M.3.1-M.3.3)**++**Goal:** Establish continuous monitoring mechanisms to track **identified, unanticipated, and emerging AI risks** throughout system operation.++DeepTeam automates ongoing assessments consistent with NIST's expectations that:++- Mechanisms and personnel are in place to **track existing and emergent risks** (M.3.1).+- Risk tracking extends to areas where **quantitative measurement is difficult** (M.3.2).+- **Feedback loops** from users and affected communities are integrated into evaluation metrics (M.3.3).++DeepTeam evaluates whether your system can:++- Detect privilege or access bypasses over time+- Track drift, misuse, or emergent vulnerabilities+- Capture user and operational feedback for remediation++**Example vulnerabilities**++- `Competition` (e.g., discreditation, market manipulation)+- `BFLA`, `BOLA` (authorization and object access controls)++**Example attacks**++- `PromptProbing`, `GrayBox`, `PromptInjection`+- `Roleplay` (Monitoring Engineer persona)++```python+from deepteam.frameworks import NIST+from deepteam import red_team+from somewhere import your_model_callback++nist = NIST(categories=["measure_3"])+attacks = nist.attacks+vulnerabilities = nist.vulnerabilities++# Modify attributes for your specific testing context if needed+risk_assessment = red_team(+ model_callback=your_model_callback,+ vulnerabilities=vulnerabilities,+ attacks=attacks+)+```++This measure reinforces NIST's principle of **adaptive oversight**, ensuring AI systems remain accountable and safe after deployment-++### Measure 4 — Impact and Transparency Assessment++**(NIST Subfunctions: M.4.1-M.4.3)**++**Goal:** Connect AI risk measurements to **business outcomes, stakeholder transparency, and lifecycle performance**.++DeepTeam's Measure 4 tests assess whether feedback mechanisms and transparency reporting accurately reflect operational trustworthiness, as defined by NIST:++- Measurement methods are **context-aware and documented** (M.4.1)+- Measurement results are **validated by domain experts and AI actors** (M.4.2)+- **Performance changes and stakeholder feedback** are incorporated into continuous improvement (M.4.3)++DeepTeam evaluates:++- Transparency and reporting effectiveness+- Accuracy of performance monitoring+- Feedback integration and risk communication++**Example vulnerability**++- `CustomVulnerability` (Transparency Assessment)++**Example attacks**++- `PromptProbing`, `BadLikertJudge`+- `Roleplay` (End User persona)++```python+from deepteam.frameworks import NIST+from deepteam import red_team+from somewhere import your_model_callback++nist = NIST(categories=["measure_4"])+attacks = nist.attacks+vulnerabilities = nist.vulnerabilities++# Modify attributes for your specific testing context if needed+risk_assessment = red_team(+ model_callback=your_model_callback,+ vulnerabilities=vulnerabilities,+ attacks=attacks+)+```++This category ensures AI systems not only measure risks effectively but also **close the feedback loop** by aligning transparency, accountability, and business value.++### Best Practices++1. **Document testing procedures** — NIST emphasizes transparency and auditability.+2. **Conduct evaluations regularly** to maintain continuous assurance.+3. **Simulate real deployment conditions** for realistic risk detection.+4. **Combine automation with human review** — automation finds exposure, humans interpret implications.+5. **Track evolving risks** using DeepTeam's analytics tools.+6. **Engage stakeholders** to contextualize and prioritize mitigation actions.++### Limitations and Human Oversight++While DeepTeam automates much of NIST-compliant testing, organizational participation remains essential for:++- Environmental and sustainability assessments+- Governance and stakeholder consultations+- Broader policy and accountability structures++Automated testing forms one component of a **comprehensive AI assurance process** — combining measurement, management, and governance.++### Learn More++- [NIST AI RMF Official Resource](https://www.nist.gov/itl/ai-risk-management-framework)+- [NIST AI RMF Playbook](https://pages.nist.gov/AIRMF/)+- [NIST AI RMF Crosswalk](https://airc.nist.gov/AI_RMF_Knowledge_Base/Crosswalks)diff --git a/docs/docs/red-teaming-owasp-top-10-for-llms.mdx b/docs/docs/frameworks-owasp-top-10-for-llms.mdxsimilarity index 99%rename from docs/docs/red-teaming-owasp-top-10-for-llms.mdxrename to docs/docs/frameworks-owasp-top-10-for-llms.mdxindex f08d9711..0f49cbc8 100644--- a/docs/docs/red-teaming-owasp-top-10-for-llms.mdx+++ b/docs/docs/frameworks-owasp-top-10-for-llms.mdx@@ -1,5 +1,5 @@----id: red-teaming-owasp-top-10-for-llms+id: frameworks-owasp-top-10-for-llmstitle: OWASP Top 10 for LLMssidebar_label: OWASP Top 10 for LLMs---diff --git a/docs/docs/red-teaming-guidelines-framework-introduction.mdx b/docs/docs/red-teaming-guidelines-framework-introduction.mdxdeleted file mode 100644index 2f26df43..00000000--- a/docs/docs/red-teaming-guidelines-framework-introduction.mdx+++ /dev/null@@ -1,7 +0,0 @@-----id: red-teaming-guidelines-framework-introduction-title: Introduction-sidebar_label: Introduction------_To be documented..._diff --git a/docs/docs/red-teaming-nist-ai-rmf.mdx b/docs/docs/red-teaming-nist-ai-rmf.mdxdeleted file mode 100644index 79da49d0..00000000--- a/docs/docs/red-teaming-nist-ai-rmf.mdx+++ /dev/null@@ -1,7 +0,0 @@-----id: red-teaming-nist-ai-rmf-title: NIST AI RMF-sidebar_label: NIST AI RMF------_To be documented..._diff --git a/docs/sidebars.js b/docs/sidebars.jsindex 39591472..e1f13e0f 100644--- a/docs/sidebars.js+++ b/docs/sidebars.js@@ -179,7 +179,14 @@ module.exports = {type: "category",label: "Guidelines & Frameworks",className: "sidebar-item-guidelines-frameworks root-sidebar-item",- items: ["red-teaming-guidelines-framework-introduction","red-teaming-owasp-top-10-for-llms", "red-teaming-nist-ai-rmf"],+ items: [+ "frameworks-introduction",+ "frameworks-owasp-top-10-for-llms",+ "frameworks-nist",+ "frameworks-mitre-atlas",+ "frameworks-beavertails",+ "frameworks-aegis",+ ],collapsed: false,},{