DeepTeam is a framework to red team LLMs and AI agents.
.
penguine33461bc07be3
diff --git a/docs/docs/red-teaming-introduction.mdx b/docs/docs/red-teaming-introduction.mdxindex 2bb5e67d..4f5a13bf 100644--- a/docs/docs/red-teaming-introduction.mdx+++ b/docs/docs/red-teaming-introduction.mdx@@ -158,7 +158,7 @@ red_team()```-For strings, you can pass a model with its provider prefix separated by a `/`. Ex: `google/gemini-2.5-flash`. The available providers that `deepteam` supports are: `openai`, `anthropic`, `google`, `xai`, `moonshotai`, `deepseek`, `ollama`.+For strings, you can pass a model with its provider prefix separated by a `/`. Ex: `google/gemini-2.5-flash`. The available providers that `deepteam` supports are: `openai`, `anthropic`, `google`, `xai`, `moonshotai`, `deepseek`, `ollama`.</TabItem><TabItem value="DeepEval model" label="DeepEval model">@@ -190,7 +190,7 @@ async def model_callback(input: str, turns: list[RTTurn] = None) -> str:# Replace this with your actual LLM application# This could be a RAG pipeline, chatbot, agent, etc.return RTTurn(- role="assistant",+ role="assistant",content="Your agent's response here...",retrieval_context=["Your retieval context here",@@ -677,40 +677,18 @@ The attacks are fed into the LLM, and the resulting responses are evaluated usinIt's worth noting that using a synthesizer model like GPT-3.5 can prove more effective than GPT-4o, as more **advanced models tend to have stricter filtering mechanisms**, which can limit the successful generation of adversarial attacks.:::-`deepteam` is highly recommended to be used in development environments, but for users in production, **Confident AI** is the recommended way to run red teaming on your applications.+`deepteam` runs locally in your development environment. [**Confident AI**](https://www.confident-ai.com) lets you run the same red teaming against production endpoints from the cloud — with scheduled runs, shareable risk reports, and no local infrastructure to maintain.-## Production Risk Assessments+## Red Teaming on the Cloud-[**Confident AI**](https://www.confident-ai.com) allows you to run red teaming on your production applications through the UI. You can connect your application to the platform by creating an AI Connection which allows Confident AI to ping your app via a HTTP request to communicate.--:::info-Production applications are the user facing applications you've deployed to the internet. It is very crucial to ensure that these applications are safe and secure against adversarial attacks and security vulnerabilities. **Confident AI** allows you to easily set up these production applications on the platform to run red teaming on them anytime and anywhere.-:::--### Connect Your Application--Confident AI has a built-in AI Connection feature that mimics the `model_callback` function in `deepteam` which allows the platform to communicate with your application. You can navigate to the "**AI Connections**" tab in your project settings and create a new AI Connection. There you can now configure the endpoint, payload, and output key path that's specific to your application.--<BlogImageDisplayer- src={ASSETS.confidentSettingsProjectAiConnection}- alt="AI Connection configuration in Confident AI"- caption="A one-time AI Connection setup to enable communication with your application."-/>--### Select a Security Framework--There are many pre-defined safety frameworks like **OWASP**, **MITRE ATLAS**, **NIST AI RMF**, and more that you can choose from on Confident AI platform. You can also curate your own custom frameworks with different vulnerabilities and attacks to suit your AI's specific needs.+Connect your application once via an AI Connection (an HTTP endpoint that mirrors the `model_callback` contract), pick a framework like OWASP or MITRE ATLAS, and run assessments on demand or on a recurring schedule. Each run produces a risk report with CVSS-style severity scores and per-test-case analysis.<BlogImageDisplayersrc={ASSETS.confidentRedTeamingChooseFramework}alt="Choosing a security framework in Confident AI"/>-### View Risk Reports--After creating a framework, you can now run a risk assessment on your application. Confident AI will generate adversarial inputs from your framework and use them against your application through the AI Connection before the entire test case is evaluated and scored.--You can also download a PDF report for each assessment for security reviews, compliance packets, and executive or stakeholder updates.+The comparison view across runs is where the value compounds — when you update a system prompt, swap models, or change retrieval logic, you can see exactly which vulnerabilities regressed and which improved. PDF exports are available for compliance documentation, security reviews, and stakeholder handoffs.<BlogImageDisplayersrc={ASSETS.confidentRedTeamingRiskAssessment}diff --git a/docs/docusaurus.config.ts b/docs/docusaurus.config.tsindex 55cac6c1..5c718ccd 100644--- a/docs/docusaurus.config.ts+++ b/docs/docusaurus.config.ts@@ -1,7 +1,7 @@-import type { Config } from '@docusaurus/types';-import type * as Preset from '@docusaurus/preset-classic';-import remarkMath from 'remark-math';-import rehypeKatex from 'rehype-katex';+import type { Config } from "@docusaurus/types";+import type * as Preset from "@docusaurus/preset-classic";+import remarkMath from "remark-math";+import rehypeKatex from "rehype-katex";const config: Config = {plugins: [@@ -24,7 +24,7 @@ const config: Config = {id: "blogs",path: "blog",routeBasePath: "blog",- blogSidebarCount: 0+ blogSidebarCount: 0,},],],@@ -37,32 +37,32 @@ const config: Config = {baseUrl: "/",onBrokenLinks: "warn",-+headTags: [{- tagName: 'meta',+ tagName: "meta",attributes: {- name: 'algolia-site-verification',- content: '0BD7859165065E7F',+ name: "algolia-site-verification",+ content: "0BD7859165065E7F",},},// GEO / SEO Organization Schema injected here{- tagName: 'script',+ tagName: "script",attributes: {- type: 'application/ld+json',+ type: "application/ld+json",},innerHTML: JSON.stringify({"@context": "https://schema.org","@type": "Organization",- "name": "DeepTeam by Confident AI",- "alternateName": "DeepTeam - The LLM Security and Red Teaming Framework",- "url": "https://trydeepteam.com",- "logo": "https://trydeepteam.com/icons/DeepTeam.svg",- "sameAs": [+ name: "DeepTeam by Confident AI",+ alternateName: "DeepTeam - The LLM Security and Red Teaming Framework",+ url: "https://trydeepteam.com",+ logo: "https://trydeepteam.com/icons/DeepTeam.svg",+ sameAs: ["https://github.com/confident-ai/deepteam",- "https://discord.gg/3SEyvpgu2f"- ]+ "https://discord.gg/3SEyvpgu2f",+ ],}),},],@@ -96,12 +96,12 @@ const config: Config = {} satisfies Preset.Options,],],- themes: ['@docusaurus/theme-mermaid'],+ themes: ["@docusaurus/theme-mermaid"],markdown: {mermaid: true,hooks: {onBrokenMarkdownLinks: "warn",- }+ },},scripts: [{@@ -139,15 +139,15 @@ const config: Config = {to: "docs/getting-started",position: "left",label: "Docs",- activeBasePath: 'docs',+ activeBasePath: "docs",},{- to: "guides/guide-custom-vulnerability",+ to: "guides/guide-agentic-ai-red-teaming",position: "left",label: "Guides",activeBasePath: "guides",},- { to: 'blog', label: 'Blog', position: 'left' },+ { to: "blog", label: "Blog", position: "left" },{href: "https://discord.gg/3SEyvpgu2f",className: "header-discord-link",@@ -190,7 +190,7 @@ const config: Config = {},{label: "Guides",- to: "/guides/guide-custom-vulnerability",+ to: "guides/guide-agentic-ai-red-teaming",},],},@@ -203,15 +203,15 @@ const config: Config = {},{label: "OWASP Top 10 for LLMs",- to: "https://www.confident-ai.com/blog/owasp-top-10-2025-for-llm-applications-risks-and-mitigation-techniques"+ to: "https://www.confident-ai.com/blog/owasp-top-10-2025-for-llm-applications-risks-and-mitigation-techniques",},{label: "The comprehensive LLM safety guide",- to: "https://www.confident-ai.com/blog/the-comprehensive-llm-safety-guide-navigate-ai-regulations-and-best-practices-for-llm-safety"+ to: "https://www.confident-ai.com/blog/the-comprehensive-llm-safety-guide-navigate-ai-regulations-and-best-practices-for-llm-safety",},{label: "LLM evaluation metrics",- to: "https://www.confident-ai.com/blog/llm-evaluation-metrics-everything-you-need-for-llm-evaluation"+ to: "https://www.confident-ai.com/blog/llm-evaluation-metrics-everything-you-need-for-llm-evaluation",},],},diff --git a/docs/guides/guide-agentic-ai-red-teaming.mdx b/docs/guides/guide-agentic-ai-red-teaming.mdxindex ffbb60e2..b4fb7092 100644--- a/docs/guides/guide-agentic-ai-red-teaming.mdx+++ b/docs/guides/guide-agentic-ai-red-teaming.mdx@@ -1,15 +1,15 @@---id: guide-agentic-ai-red-teaming-title: Complete Agentic AI red teaming guide+title: Complete Guide to Agentic AI Red Teamingsidebar_label: AI Agents---import BlogImageDisplayer from "@site/src/components/BlogImageDisplayer";import { ASSETS } from "@site/src/assets";-AI agents introduce a fundamentally different attack surface compared to agentic RAG or conversational agent systems. An AI agent does not just generate text—it reasons, plans, calls tools, delegates to sub-agents, and takes actions with real-world consequences. A compromised agent can delete files, make unauthorized API calls, leak credentials through tool descriptions, or drift from its assigned objectives without producing any visible error. Traditional prompt-level red teaming is necessary but insufficient for agentic AI systems that operate autonomously.+AI agents don't just generate text — they reason, plan, call tools, and take actions with real-world consequences. A compromised agent can delete files, leak credentials, or make unauthorized API calls without producing any visible error.-This guide explains agentic AI red teaming with [DeepTeam](https://github.com/confident-ai/deepteam) — how to systematically red team AI agents that call tools, delegate tasks, and act on real infrastructure. The running example is an **internal DevOps agent** — a system that monitors infrastructure, creates tickets, runs diagnostic commands, and escalates incidents. The same approach applies to any tool-calling AI agent, multi-agent pipeline, or autonomous agentic workflow.+This guide covers agentic AI red teaming with [DeepTeam](https://github.com/confident-ai/deepteam). The running example is an **internal DevOps agent** that monitors infrastructure, creates tickets, and escalates incidents — but the same approach applies to any tool-calling agent or multi-agent pipeline.:::noteThis guide focuses on agentic red teaming for tool-calling AI agents. For RAG-based agents, use the [agentic RAG guide](/guides/guide-red-teaming-agentic-rag). For conversational agents, use the [conversational agents guide](/guides/guide-red-teaming-conversational-agents).@@ -30,28 +30,19 @@ A compromised AI agent won't throw errors. It will delete files, leak credentialTraditional LLM applications have one attack surface: the prompt-response interface. AI agents have three, and each one can be compromised independently.```mermaid-flowchart TB- subgraph Reasoning["Reasoning Layer"]- direction LR- R1["Planning & goal decomposition"]- R2["Tool selection decisions"]- end- subgraph Action["Action Layer"]- direction LR- A1["Tool invocation"]- A2["Parameter generation"]- A3["Output processing"]- end- subgraph InterAgent["Inter-Agent Communication"]- direction LR- IA1["Delegation messages"]- IA2["Shared context"]- end-- User -->|"Adversarial input"| Reasoning- Reasoning -->|"Tool calls"| Action- Action -->|"Results"| Reasoning- Reasoning <-->|"Messages"| InterAgent+sequenceDiagram+ participant User+ participant Reasoning as Reasoning Layer+ participant Action as Action Layer+ participant InterAgent as Inter-Agent Comm.++ User->>Reasoning: Adversarial input+ Reasoning->>Reasoning: Planning & goal decomposition+ Reasoning->>Action: Tool calls+ Action->>Action: Tool invocation & parameter generation+ Action-->>Reasoning: Results & output processing+ Reasoning->>InterAgent: Delegation messages+ InterAgent-->>Reasoning: Shared context```**The reasoning layer** is where the agent decides _what_ to do. An attacker who compromises reasoning can redirect the agent's goals, cause it to ignore its operational boundaries, or make it "forget" constraints from its system prompt. This is where vulnerabilities like [`GoalTheft`](/docs/red-teaming-vulnerabilities-goal-theft) and [`AutonomousAgentDrift`](/docs/red-teaming-vulnerabilities-autonomous-agent-drift) operate.@@ -72,12 +63,18 @@ Understanding which layers your agent exposes helps you prioritize the right vulThe agentic red teaming process follows the same structure as any DeepTeam assessment, with two important additions: the AI agent's tool-calling behavior must be captured for evaluation, and the vulnerability surface extends beyond language output to include actions, permissions, and inter-agent communication.```mermaid-flowchart LR- A["Map agent\ncapabilities"] --> B["Select\nvulnerabilities"]- B --> C["Choose\nattacks"]- C --> D["Integrate agent\nin model_callback"]- D --> E["Run red_team()\n& analyze"]- E -->|"Fix & retest"| B+sequenceDiagram+ participant You+ participant DeepTeam+ participant Agent as Your AI Agent++ You->>You: Map agent capabilities+ You->>DeepTeam: Select vulnerabilities & attacks+ You->>Agent: Integrate agent in model_callback+ DeepTeam->>Agent: Run red_team()+ Agent-->>DeepTeam: Agent responses + tool calls+ DeepTeam-->>You: Results & analysis+ You-->>DeepTeam: Fix & retest```1. **Map the AI agent's capabilities** — Enumerate every tool the agent can call, every external system it can reach, and every delegation path to other agents. This defines the blast radius of a successful attack.diff --git a/docs/guides/guide-cli-yaml.mdx b/docs/guides/guide-cli-yaml.mdxindex 5be5b05f..a62ebae5 100644--- a/docs/guides/guide-cli-yaml.mdx+++ b/docs/guides/guide-cli-yaml.mdx@@ -1,6 +1,6 @@---id: guide-cli-yaml-title: Red teaming from the CLI+title: Red Teaming from the CLIsidebar_label: CLI & YAML Configs---diff --git a/docs/guides/guide-custom-attacks.mdx b/docs/guides/guide-custom-attacks.mdxindex de462a83..9a666a64 100644--- a/docs/guides/guide-custom-attacks.mdx+++ b/docs/guides/guide-custom-attacks.mdx@@ -179,10 +179,15 @@ for test_case in test_cases:The base attack in your test cases are first enhanced using roleplay, this attack now disguised as roleplay is then enhanced using prompt injection again. This makes the base attack already sophisticated enough, however, the prompt injected attack is then encoded using the leetspeak technique making this attack even more powerful.```mermaid-flowchart LR- BA[Base attack] --> RP[Roleplay]- RP --> PI[Prompt Injection]- PI --> LS[Leetspeak]+sequenceDiagram+ participant BA as Base Attack+ participant RP as Roleplay+ participant PI as Prompt Injection+ participant LS as Leetspeak++ BA->>RP: Enhance+ RP->>PI: Enhance+ PI->>LS: Enhance```This allows you to _test, iterate, and improve_ attacks to find hidden vulnerabilities in your LLM applications.@@ -292,10 +297,15 @@ The `_get_turns` method has **ONE** mandatory and **THREE** optional parameters:The base attack in your test cases are first enhanced using roleplay, this attack now disguised as roleplay is then used as the first turn to progress the attack from. After getting new turns that are already _ready to be evaluated_ (Most LLM applications already fail here). You can still use these new turns to progress them even further with another multi-turn progression technique.```mermaid-flowchart LR- BA[Base attack] --> RP[Roleplay]- RP --> LJ[Linear jailbreaking progressions ]- LJ --> SJ[Sequential Jailbreaking progressions ]+sequenceDiagram+ participant BA as Base Attack+ participant RP as Roleplay+ participant LJ as Linear Jailbreaking+ participant SJ as Sequential Jailbreaking++ BA->>RP: Enhance+ RP->>LJ: Progress+ LJ->>SJ: Progress```Now, [evaluate these test cases](#evaluating-test-cases) to see if your LLM application passes the vulnerability you've chosen when using multi-turn progressions.diff --git a/docs/guides/guide-custom-red-teaming-pipelines.mdx b/docs/guides/guide-custom-red-teaming-pipelines.mdxindex 2699c9e8..f7f3aa36 100644--- a/docs/guides/guide-custom-red-teaming-pipelines.mdx+++ b/docs/guides/guide-custom-red-teaming-pipelines.mdx@@ -22,10 +22,15 @@ Red teaming usually starts with a vulnerability. It allows us to choose the basiBelow is an image that shows the flow of a red teaming process:```mermaid-graph LR- BV[Base vulnerability] -->|Baseline attack| AA[Adversarial attack]- AA -->|Test cases| VM[Vulnerability metric]- VM -->|Metric results| RA[Risk Assessment]+sequenceDiagram+ participant BV as Base Vulnerability+ participant AA as Adversarial Attack+ participant VM as Vulnerability Metric+ participant RA as Risk Assessment++ BV->>AA: Baseline attack+ AA->>VM: Test cases+ VM->>RA: Metric results```The base vulnerabilities generates harmful prompts called baseline attacks, these attacks are then enhanced or progressed using the adversarial attack techniques, which allow us to create test cases that can be used to evaluate whether your LLM application has successfully avoided producing harmful outputs.diff --git a/docs/guides/guide-deploying-guardrails.mdx b/docs/guides/guide-deploying-guardrails.mdxindex 63e5aec9..906ab81f 100644--- a/docs/guides/guide-deploying-guardrails.mdx+++ b/docs/guides/guide-deploying-guardrails.mdx@@ -1,6 +1,6 @@---id: guide-deploying-guardrails-title: Deploying guardrails in production+title: Deploying Guardrails in Productionsidebar_label: Guardrails in Production---diff --git a/docs/guides/guide-red-teaming-agentic-rag.mdx b/docs/guides/guide-red-teaming-agentic-rag.mdxindex 9dea9659..3c52325b 100644--- a/docs/guides/guide-red-teaming-agentic-rag.mdx+++ b/docs/guides/guide-red-teaming-agentic-rag.mdx@@ -1,6 +1,6 @@---id: guide-red-teaming-agentic-rag-title: Red teaming agentic RAG pipelines+title: Red Teaming Agentic RAG Pipelinessidebar_label: Agentic RAG---@@ -119,14 +119,17 @@ The table below summarizes the attack techniques and where they operate in the R| `Leetspeak` / `Base64` / `ROT13` | Input filter layer | Encodes harmful content to bypass keyword-based filters | Medium — tests filter robustness, not agent robustness |```mermaid-flowchart LR- U["User query"] --> R["Retriever"]- R -->|"Retrieved chunks\n(attack surface)"| A["Agent"]- A --> O["Response"]-- PI["PromptInjection\nRoleplay"] -.->|"Target"| U- CF["ContextFlooding\nSyntheticContextInjection"] -.->|"Target"| R- ENC["Leetspeak / Base64\nROT13"] -.->|"Target"| U+sequenceDiagram+ participant User+ participant Retriever+ participant Agent+ participant Output as Response++ User->>Retriever: User query+ Note over User: Targets: PromptInjection,<br/>Roleplay, Leetspeak,<br/>Base64, ROT13+ Retriever->>Agent: Retrieved chunks (attack surface)+ Note over Retriever: Targets: ContextFlooding,<br/>SyntheticContextInjection+ Agent->>Output: Response```:::tip@@ -236,27 +239,17 @@ result = await cross_context.assess(model_callback=model_callback)The workflow: run the full assessment, find the weak spots, use `assess()` to measure how consistent the failure is, fix the root cause, then re-run to confirm.-## Production Risk Assessments+## Bringing Risk Assessments to Your CISO-[Confident AI](https://www.confident-ai.com) lets you red team production agentic RAG pipelines directly from its platform—no local code needed. Connect your RAG-based agent via an AI Connection, pick a framework (OWASP, MITRE, NIST, or custom), and run assessments on demand or on a schedule.+RAG pipelines break in ways that are hard to catch locally — a retrieval index update surfaces new chunks the agent has never seen, a system prompt tweak opens a context injection path, or a model swap changes which attacks succeed. [Confident AI](https://www.confident-ai.com) lets you schedule recurring red teaming against your production RAG endpoint so these regressions surface before users find them.-<BlogImageDisplayer- src={ASSETS.confidentSettingsProjectAiConnection}- alt="AI Connection configuration in Confident AI"- caption="A one-time setup to connect your RAG-based agent for on-demand red teaming."-/>--Schedule assessments to run regularly so you catch regressions when prompts, indexes, or models change. Each run produces a risk report with per-test-case analysis. PDF exports are available for security reviews and stakeholders.+Connect your RAG-based agent via an AI Connection — point it at your HTTP endpoint and map the request/response schema once. Then select a framework (OWASP, MITRE, NIST) and schedule daily or weekly runs. Each assessment produces a risk report with per-test-case pass/fail analysis and CVSS-style severity scores, so you can see exactly which retrieval-context attacks succeeded and which vulnerability categories regressed since the last run.<BlogImageDisplayersrc={ASSETS.confidentRedTeamingRiskAssessment}alt="Risk assessment dashboard in Confident AI"/>-:::info-[Confident AI](https://app.confident-ai.com) has a free tier to get started—no credit card required.-:::-## What to Do NextRed teaming agentic RAG isn't a one-time activity. Every time you update a system prompt, change the retrieval index, or swap the underlying model, the attack surface shifts.diff --git a/docs/guides/guide-red-teaming-anthropic.mdx b/docs/guides/guide-red-teaming-anthropic.mdxnew file mode 100644index 00000000..6696ae1c--- /dev/null+++ b/docs/guides/guide-red-teaming-anthropic.mdx@@ -0,0 +1,168 @@+---+id: guide-red-teaming-anthropic+title: Red Teaming Anthropic Claude+sidebar_label: Anthropic Claude+---++Anthropic's Claude models (Sonnet, Opus, Haiku) are trained with Constitutional AI (CAI) — a self-critique alignment method where the model evaluates its own outputs against a set of principles before responding. This produces safety behavior that is structurally different from RLHF-only models: Claude tends to reason about _why_ a request is harmful rather than pattern-matching on surface-level triggers. The result is stronger resistance to direct jailbreaks but a different set of weaknesses — particularly around multi-turn consistency manipulation and appeals to the model's own helpfulness principles.++This guide covers red teaming Anthropic Claude with [DeepTeam](https://github.com/confident-ai/deepteam).++:::note+For general model red teaming methodology, vulnerability selection guidance, and framework-based assessments, see the [foundational models guide](/guides/guide-red-teaming-models).+:::++## YAML Quickstart++Claude requires a custom callback since DeepTeam's native YAML target supports OpenAI models. Create the callback first:++```python title="my_model.py"+import anthropic++client = anthropic.Anthropic()++async def model_callback(input: str) -> str:+ message = client.messages.create(+ model="claude-sonnet-4-20250514",+ max_tokens=1024,+ messages=[{"role": "user", "content": input}],+ )+ return message.content[0].text+```++```yaml title="red-team-claude.yaml"+target:+ purpose: "A financial advisory assistant for retail banking customers"+ callback:+ file: "my_model.py"+ function: "model_callback"++default_vulnerabilities:+ - name: "Toxicity"+ - name: "Bias"+ types: ["race", "gender", "religion"]+ - name: "PromptLeakage"+ - name: "IllegalActivity"++attacks:+ - name: "Roleplay"+ weight: 3+ - name: "PromptInjection"+ weight: 2+ - name: "Leetspeak"+ - name: "LinearJailbreaking"+ num_turns: 5+```++```bash+deepteam run red-team-claude.yaml+```++## Python Assessment++```python+from deepteam import red_team+from deepteam.vulnerabilities import Toxicity, Bias, PromptLeakage, IllegalActivity, Misinformation+from deepteam.attacks.single_turn import PromptInjection, Roleplay, Leetspeak+from deepteam.attacks.multi_turn import LinearJailbreaking, CrescendoJailbreaking+import anthropic++client = anthropic.Anthropic()++async def model_callback(input: str) -> str:+ message = client.messages.create(+ model="claude-sonnet-4-20250514",+ max_tokens=1024,+ messages=[{"role": "user", "content": input}],+ )+ return message.content[0].text++red_team(+ model_callback=model_callback,+ target_purpose="A financial advisory assistant for retail banking customers",+ vulnerabilities=[+ Toxicity(), Bias(), PromptLeakage(),+ IllegalActivity(), Misinformation(),+ ],+ attacks=[+ PromptInjection(), Roleplay(), Leetspeak(),+ LinearJailbreaking(), CrescendoJailbreaking(),+ ],+ attacks_per_vulnerability_type=5,+)+```++## Multi-Turn Testing++Claude's Constitutional AI training makes it particularly interesting to test across multiple turns. The model's self-critique mechanism can be worn down through gradual context manipulation — where each turn subtly shifts the model's assessment of what constitutes a helpful response.++```python+from deepteam.test_case import RTTurn++async def model_callback(input: str, turns: list[RTTurn] = None) -> str:+ messages = []+ for turn in (turns or []):+ messages.append({"role": turn.role, "content": turn.content})+ messages.append({"role": "user", "content": input})++ message = client.messages.create(+ model="claude-sonnet-4-20250514",+ max_tokens=1024,+ messages=messages,+ )+ return message.content[0].text+```++## Claude-Specific Considerations++### Safety Training Characteristics++Claude's Constitutional AI alignment produces distinctive behaviors that affect red teaming strategy:++- **Principled refusal** — Claude refuses harmful requests by reasoning about principles rather than pattern-matching keywords. This makes it more resistant to simple encoding attacks (`Leetspeak`, `ROT13`) than RLHF-only models, because the model evaluates semantic intent after decoding.+- **Helpfulness tension** — Claude's training heavily weights being helpful, thorough, and honest. Attacks that frame harmful requests as educational, research-oriented, or safety-focused exploit this tension — the model's helpfulness principle can override its harm-avoidance principle.+- **Multi-turn consistency pressure** — Once Claude commits to a line of reasoning, it tends to remain consistent. An attacker who establishes a benign framing in early turns can gradually escalate, leveraging the model's own prior responses as implicit permission.+- **System prompt adherence** — Claude treats system prompts as strong constraints relative to other models. `PromptLeakage` attacks are less likely to succeed through direct elicitation but can still work through indirect approaches (summarization requests, translation tasks).++### Testing with System Prompts++Claude's system prompt is passed as a separate parameter, not a message:++```python+async def model_callback(input: str) -> str:+ message = client.messages.create(+ model="claude-sonnet-4-20250514",+ max_tokens=1024,+ system="You are a financial advisor. Never provide specific investment recommendations.",+ messages=[{"role": "user", "content": input}],+ )+ return message.content[0].text+```++Test with your production system prompt to evaluate real-world safety boundaries.++### Recommended Attack Strategy++| Attack | Effectiveness Against Claude | Why |+|---|---|---|+| `Roleplay` | High | Exploits helpfulness-vs-safety tension through fictional framing |+| `LinearJailbreaking` | High | Gradual multi-turn escalation wears down principled refusal |+| `CrescendoJailbreaking` | High | Context accumulation exploits consistency pressure |+| `PromptInjection` | Medium | Claude evaluates intent semantically, not just pattern-matching |+| `Leetspeak` / `ROT13` | Lower | CAI self-critique catches encoded harmful content more reliably |++### Recommended Vulnerability Priority++| Use Case | Priority Vulnerabilities | Why |+|---|---|---|+| Customer-facing | `Toxicity`, `Bias`, `PIILeakage` | Brand safety, user trust |+| Regulated industry | `IllegalActivity`, `Misinformation`, `PersonalSafety` | Compliance, liability |+| Internal tooling | `PromptLeakage`, `SQLInjection`, `ShellInjection` | Infrastructure boundaries |+| Content generation | `Bias`, `Misinformation`, `IntellectualProperty` | Output quality, legal risk |++## What to Do Next++- **Prioritize multi-turn attacks** — Claude's primary weakness is gradual escalation, not single-shot jailbreaks. Run `LinearJailbreaking` and `CrescendoJailbreaking` with higher turn counts (`num_turns: 5+`).+- **Compare Sonnet vs. Haiku** — Smaller models (Haiku) may have weaker safety boundaries than larger ones (Sonnet, Opus). Run the same assessment across model sizes.+- **Test against frameworks** — Use `OWASPTop10()` for standardized coverage. See the [safety frameworks guide](/guides/guide-safety-frameworks).+- **Move to application-level testing** — Once the model passes, test it in context with the [AI agents](/guides/guide-agentic-ai-red-teaming), [conversational agents](/guides/guide-red-teaming-conversational-agents), or [agentic RAG](/guides/guide-red-teaming-agentic-rag) guides.diff --git a/docs/guides/guide-red-teaming-conversational-agents.mdx b/docs/guides/guide-red-teaming-conversational-agents.mdxindex 47b5dcc8..cdcfdb1b 100644--- a/docs/guides/guide-red-teaming-conversational-agents.mdx+++ b/docs/guides/guide-red-teaming-conversational-agents.mdx@@ -1,6 +1,6 @@---id: guide-red-teaming-conversational-agents-title: Red teaming conversational AI agents+title: Red Teaming Conversational AI Agentssidebar_label: Conversational Agents---@@ -30,20 +30,19 @@ An agent that's perfectly safe on any single message can still fail catastrophicA single-turn attack gives you one shot — the prompt either breaks through or it doesn't. Multi-turn attacks are fundamentally more powerful because they exploit three properties of conversation that don't exist in single-turn interactions.```mermaid-flowchart LR- subgraph Turn1["Turn 1"]- U1["Benign probe"] --> A1["Agent responds"]- end- subgraph Turn2["Turn 2"]- U2["Build on response"] --> A2["Agent stays consistent"]- end- subgraph Turn3["Turn 3"]- U3["Reframe intent"] --> A3["Agent adapts"]- end- subgraph TurnN["Turn N"]- UN["Escalate to target"] --> AN["Agent compromised"]- end- Turn1 --> Turn2 --> Turn3 -->|"..."| TurnN+sequenceDiagram+ participant Attacker+ participant Agent as Conversational Agent++ Attacker->>Agent: Turn 1: Benign probe+ Agent-->>Attacker: Agent responds normally+ Attacker->>Agent: Turn 2: Build on response+ Agent-->>Attacker: Agent stays consistent+ Attacker->>Agent: Turn 3: Reframe intent+ Agent-->>Attacker: Agent adapts+ Note over Attacker,Agent: ... continued escalation ...+ Attacker->>Agent: Turn N: Escalate to target+ Agent-->>Attacker: Agent compromised```**Consistency pressure.** Once a conversational agent commits to a tone, framing, or line of reasoning, it tends to maintain that trajectory. An attacker who establishes a benign conversational context in early turns can gradually steer the agent into territory it would refuse to enter from a cold start. The agent's own prior responses become leverage.@@ -59,12 +58,18 @@ These dynamics mean that a conversational agent can be perfectly safe on any sinThe red teaming process for a conversational AI agent follows the same structure as any DeepTeam assessment, with one important distinction: multi-turn attacks require a live connection to the target agent. The attack logic reads each assistant response and adapts the next user message based on what it observes. This means testing cannot use pre-recorded outputs—the agent must respond in real time.```mermaid-flowchart LR- A["Identify\nfailure modes"] --> B["Select\nvulnerabilities"]- B --> C["Choose multi-turn\nattack strategies"]- C --> D["Integrate agent\nin model_callback"]- D --> E["Run red_team()\n& analyze"]- E -->|"Fix & retest"| B+sequenceDiagram+ participant You+ participant DeepTeam+ participant Agent as Your Conversational Agent++ You->>You: Identify failure modes+ You->>DeepTeam: Select vulnerabilities & attacks+ You->>Agent: Integrate agent in model_callback+ DeepTeam->>Agent: Run red_team()+ Agent-->>DeepTeam: Agent responses+ DeepTeam-->>You: Results & analysis+ You-->>DeepTeam: Fix & retest```1. **Identify failure modes** — Determine what the conversational agent must never say, do, or facilitate based on its function, user base, and regulatory context.@@ -401,47 +406,18 @@ result = await prompt_leakage.assess(model_callback=model_callback)The recommended workflow: run the full assessment → identify the vulnerabilities with the lowest pass rates → use `assess()` to stress-test those areas and measure failure consistency → implement fixes → re-run the full assessment to verify the resolution.-## Production Risk Assessments+## Bringing Results to Your CISO-[Confident AI](https://www.confident-ai.com) allows teams to run red teaming against production conversational AI agents through its platform, without requiring local code execution. Agents are connected through an AI Connection, which allows Confident AI to communicate with the conversational agent via HTTP requests—eliminating the need to manage test environments separately from production.+Conversational agents are particularly sensitive to regressions — a system prompt update, a model swap, or a change in conversation history handling can re-open vulnerabilities that previously passed. Running `red_team()` locally catches issues during development, but production conversational agents need continuous monitoring.-:::info-Confident AI mirrors the `model_callback` contract in code: it sends adversarial inputs to the configured endpoint, receives responses, and evaluates them against the selected vulnerability criteria. The setup is a one-time configuration.-:::--### Connect Your Agent--1. Navigate to the "**AI Connections**" tab in the project settings.-2. Create a new AI Connection.-3. Configure the endpoint URL, request payload shape, and the output key path that maps to the agent's reply.--<BlogImageDisplayer- src={ASSETS.confidentSettingsProjectAiConnection}- alt="AI Connection configuration in Confident AI"- caption="A one-time AI Connection setup to enable communication with your conversational agent."-/>--### Select a Security Framework--The platform supports OWASP, MITRE ATLAS, NIST AI RMF, and custom frameworks. For a customer support conversational agent, the OWASP Top 10 for LLMs provides a well-scoped starting point that covers the primary risk categories relevant to conversational AI agents.--<BlogImageDisplayer- src={ASSETS.confidentRedTeamingChooseFramework}- alt="Choosing a security framework in Confident AI"-/>--### Schedule Assessments and Access Reports--Assessments can be scheduled for daily or weekly execution to detect regressions proactively as the agent's system prompt, model version, or retrieval configuration changes. Each completed run produces a detailed risk report with CVSS-style severity scores and per-test-case analysis. PDF reports can be downloaded for security reviews, compliance documentation, and stakeholder handoffs.+[Confident AI](https://www.confident-ai.com) lets you schedule automated red teaming against your production conversational agent. Connect it via an AI Connection (a one-time HTTP endpoint configuration), select a framework like OWASP Top 10, and schedule daily or weekly runs. The platform sends multi-turn adversarial inputs to your endpoint, evaluates responses against vulnerability criteria, and produces risk reports with per-test-case analysis and CVSS-style severity scores.<BlogImageDisplayersrc={ASSETS.confidentRedTeamingRiskAssessment}alt="Risk assessment dashboard in Confident AI"/>-:::info-Set up [Confident AI](https://app.confident-ai.com) and run the first risk assessment in minutes. **The platform offers a free tier to get started.** _(No credit card required)_-:::+The comparison view across runs is especially valuable for conversational agents — when you update a system prompt or switch models, you can see exactly which multi-turn attack patterns started succeeding that previously failed. PDF exports are available for compliance documentation and stakeholder reviews.## What to Do Nextdiff --git a/docs/guides/guide-red-teaming-fine-tuned.mdx b/docs/guides/guide-red-teaming-fine-tuned.mdxnew file mode 100644index 00000000..57c4af21--- /dev/null+++ b/docs/guides/guide-red-teaming-fine-tuned.mdx@@ -0,0 +1,275 @@+---+id: guide-red-teaming-fine-tuned+title: Red Teaming Fine-Tuned Models+sidebar_label: Fine-Tuned Models+---++Fine-tuning a model changes its safety properties. This is true whether you fine-tuned intentionally for safety, fine-tuned on domain data without thinking about safety at all, or used a community adapter that someone else trained. The base model's alignment is a product of its original safety training — and fine-tuning overwrites, dilutes, or disrupts that training in ways that are not always predictable.++A model that passes red teaming before fine-tuning may fail afterward. Conversely, safety-focused fine-tuning may introduce new refusal patterns that create a different set of edge cases. Either way, the fine-tuned model's safety properties are _not_ the same as the base model's, and must be tested independently.++This guide covers red teaming methodology for fine-tuned models with [DeepTeam](https://github.com/confident-ai/deepteam).++:::note+For general model red teaming methodology, see the [foundational models guide](/guides/guide-red-teaming-models). For deployment-level concerns with open-weight models, see the [open-weight models guide](/guides/guide-red-teaming-open-weight).+:::++## How Fine-Tuning Affects Safety++### Safety Degradation from Domain Fine-Tuning++The most common scenario: you fine-tune a model on domain-specific data (medical records, legal documents, customer transcripts, code) to improve task performance. The training data is benign. But the fine-tuning process still degrades safety:++- **Catastrophic forgetting** — Safety training occupies specific weight patterns. Fine-tuning on new data shifts these weights, and safety behaviors are disproportionately affected because they are often in tension with the model's primary objective (being helpful and following instructions).+- **Distribution shift** — The model learns to prioritize the fine-tuning domain's language patterns. If the fine-tuning data includes content that the base model would have refused (e.g., medical descriptions of injuries, legal descriptions of crimes), the model learns that this content is acceptable to produce.+- **Refusal boundary erosion** — Each fine-tuning step that rewards the model for producing domain content without refusal weakens the model's general ability to refuse. The model learns that "not refusing" is correct behavior.++### Safety Changes from LoRA and Adapters++Parameter-efficient fine-tuning (LoRA, QLoRA, prefix tuning) modifies fewer weights than full fine-tuning, but safety degradation still occurs:++- LoRA adapters can override safety-critical weight patterns even with low rank.+- Merged adapters (where the LoRA weights are folded into the base model) produce unpredictable safety interactions.+- Stacking multiple adapters compounds safety degradation.++### Intentional Safety Fine-Tuning++Fine-tuning specifically for safety (adding refusal examples, RLHF, DPO with safety-focused preference data) can improve safety — but introduces its own edge cases:++- Over-refusal on benign content the model incorrectly classifies as harmful.+- Inconsistent refusal boundaries where the model refuses some phrasings but not semantic equivalents.+- Brittleness — safety fine-tuning may work for the training distribution but fail on adversarial inputs that differ from the safety training examples.++## Methodology++Fine-tuned model red teaming requires a comparative approach:++```mermaid+sequenceDiagram+ participant You+ participant DeepTeam+ participant Base as Base Model+ participant FT as Fine-Tuned Model++ You->>DeepTeam: Define assessment+ DeepTeam->>Base: Run red_team()+ Base-->>DeepTeam: Baseline results+ DeepTeam->>FT: Run red_team()+ FT-->>DeepTeam: Fine-tuned results+ DeepTeam-->>You: Compare results+ You->>You: Identify safety regression+```++1. **Establish a baseline** — Run the same assessment against the base model to understand its safety properties.+2. **Test the fine-tuned model** — Run the identical assessment against the fine-tuned model.+3. **Compare results** — Identify which vulnerabilities degraded, which improved, and which are unchanged.+4. **Investigate regressions** — For each vulnerability that got worse, determine whether the degradation is from training data influence, catastrophic forgetting, or distribution shift.+5. **Fix and retest** — Apply guardrails, adjust fine-tuning data, or add safety examples, then retest.++## Running a Comparative Assessment++### Base Model Callback++```python+from openai import OpenAI++base_client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")++async def base_callback(input: str) -> str:+ response = base_client.chat.completions.create(+ model="meta-llama/Llama-3.1-8B-Instruct",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++### Fine-Tuned Model Callback++```python+ft_client = OpenAI(base_url="http://localhost:8001/v1", api_key="dummy")++async def finetuned_callback(input: str) -> str:+ response = ft_client.chat.completions.create(+ model="my-org/Llama-3.1-8B-medical-v2",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++### Run Both Assessments++```python+from deepteam import red_team+from deepteam.vulnerabilities import (+ Toxicity, Bias, PromptLeakage, PIILeakage,+ IllegalActivity, Misinformation, PersonalSafety+)+from deepteam.attacks.single_turn import PromptInjection, Roleplay, Leetspeak+from deepteam.attacks.multi_turn import LinearJailbreaking++VULNERABILITIES = [+ Toxicity(), Bias(), PromptLeakage(), PIILeakage(),+ IllegalActivity(), Misinformation(), PersonalSafety(),+]+ATTACKS = [+ PromptInjection(), Roleplay(), Leetspeak(), LinearJailbreaking(),+]++print("=== Base Model ===")+red_team(+ model_callback=base_callback,+ target_purpose="A medical information assistant",+ vulnerabilities=VULNERABILITIES,+ attacks=ATTACKS,+ attacks_per_vulnerability_type=5,+)++print("=== Fine-Tuned Model ===")+red_team(+ model_callback=finetuned_callback,+ target_purpose="A medical information assistant",+ vulnerabilities=VULNERABILITIES,+ attacks=ATTACKS,+ attacks_per_vulnerability_type=5,+)+```++Compare pass rates per vulnerability. A drop in any category indicates fine-tuning-induced safety regression.++## Fine-Tuning-Specific Vulnerabilities++Beyond standard vulnerabilities, fine-tuned models have additional failure modes:++### Training Data Leakage++Fine-tuned models can memorize and reproduce training data. If the fine-tuning dataset contains sensitive information (PII, proprietary data, internal documents), the model may leak it:++```python+from deepteam.vulnerabilities import PIILeakage, PromptLeakage++red_team(+ model_callback=finetuned_callback,+ target_purpose="A medical information assistant trained on patient records",+ vulnerabilities=[PIILeakage(), PromptLeakage()],+ attacks=[PromptInjection(), Roleplay()],+ attacks_per_vulnerability_type=10,+)+```++Use higher `attacks_per_vulnerability_type` for training data leakage — memorization is inconsistent and may only surface with specific prompt patterns.++### Domain-Induced Permissiveness++A model fine-tuned on medical data may become more willing to discuss symptoms, medications, and procedures in ways that cross into providing unauthorized medical advice. A model fine-tuned on legal data may produce content that constitutes legal counsel. Test whether the model maintains appropriate boundaries for its deployment context:++```python+from deepteam.vulnerabilities import PersonalSafety, Misinformation, IllegalActivity++red_team(+ model_callback=finetuned_callback,+ target_purpose="A medical information assistant (not a licensed provider)",+ vulnerabilities=[PersonalSafety(), Misinformation(), IllegalActivity()],+ attacks=[PromptInjection(), Roleplay()],+ attacks_per_vulnerability_type=5,+)+```++### Refusal Regression++Test whether the fine-tuned model refuses content that the base model would refuse:++```python+from deepteam.vulnerabilities import Toxicity, Bias, IllegalActivity++REGRESSION_VULNS = [Toxicity(), Bias(), IllegalActivity()]+REGRESSION_ATTACKS = [PromptInjection(), Roleplay(), Leetspeak()]++print("=== Base Model Refusal ===")+red_team(+ model_callback=base_callback,+ target_purpose="A general assistant",+ vulnerabilities=REGRESSION_VULNS,+ attacks=REGRESSION_ATTACKS,+ attacks_per_vulnerability_type=5,+)++print("=== Fine-Tuned Model Refusal ===")+red_team(+ model_callback=finetuned_callback,+ target_purpose="A general assistant",+ vulnerabilities=REGRESSION_VULNS,+ attacks=REGRESSION_ATTACKS,+ attacks_per_vulnerability_type=5,+)+```++## Testing LoRA and Adapter Configurations++### Single Adapter++```python+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")++async def model_callback(input: str) -> str:+ response = client.chat.completions.create(+ model="base-model+my-lora-adapter",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++### Merged Adapter++If you've merged a LoRA adapter into the base weights, test the merged model directly:++```python+async def model_callback(input: str) -> str:+ response = client.chat.completions.create(+ model="my-org/llama-3.1-8b-merged-medical",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++Compare results between the adapter-applied and merged versions — merging can produce different safety properties than runtime adapter application.++## Continuous Testing++Fine-tuning is iterative. Each training run, data update, or adapter change can shift safety properties. Integrate red teaming into your fine-tuning pipeline:++```yaml title=".github/workflows/fine-tune-safety.yml"+name: Fine-Tune Safety Check+on:+ workflow_dispatch:+ push:+ paths:+ - "models/**"+ - "training/**"++jobs:+ safety-check:+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4+ - uses: actions/setup-python@v5+ with:+ python-version: "3.11"+ - run: pip install deepteam+ - run: deepteam run safety-check.yaml -o results+ env:+ OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}+ - uses: actions/upload-artifact@v4+ with:+ name: safety-results+ path: results/+```++## What to Do Next++- **Always compare against the base model** — Fine-tuning safety regressions are only visible in comparison.+- **Test after every training run** — Safety properties change with each fine-tuning iteration.+- **Deploy guardrails** — Fine-tuned models have less predictable safety boundaries. Application-level [guardrails](/guides/guide-deploying-guardrails) compensate for fine-tuning-induced regressions.+- **Test against frameworks** — Use `OWASPTop10()` for standardized coverage. See the [safety frameworks guide](/guides/guide-safety-frameworks).+- **Increase attack density for PII** — Fine-tuned models may memorize training data. Use higher `attacks_per_vulnerability_type` for `PIILeakage` and `PromptLeakage`.diff --git a/docs/guides/guide-red-teaming-gemini.mdx b/docs/guides/guide-red-teaming-gemini.mdxnew file mode 100644index 00000000..923a49a2--- /dev/null+++ b/docs/guides/guide-red-teaming-gemini.mdx@@ -0,0 +1,172 @@+---+id: guide-red-teaming-gemini+title: Red Teaming Google Gemini+sidebar_label: Google Gemini+---++Google Gemini models (Gemini Pro, Flash, Ultra) have a unique safety architecture: configurable safety settings that act as tunable filters on top of the model's built-in alignment. Developers can adjust thresholds for harassment, hate speech, sexually explicit content, and dangerous content from `BLOCK_NONE` to `BLOCK_ONLY_HIGH` to `BLOCK_LOW_AND_ABOVE`. This means the same model can behave very differently depending on configuration — and red teaming must test the specific configuration deployed in production.++This guide covers red teaming Google Gemini with [DeepTeam](https://github.com/confident-ai/deepteam).++:::note+For general model red teaming methodology, vulnerability selection guidance, and framework-based assessments, see the [foundational models guide](/guides/guide-red-teaming-models).+:::++## YAML Quickstart++Gemini requires a custom callback. Create it using the Google AI SDK:++```python title="my_model.py"+import google.generativeai as genai++genai.configure(api_key="YOUR_API_KEY")+model = genai.GenerativeModel("gemini-2.0-flash")++async def model_callback(input: str) -> str:+ response = model.generate_content(input)+ return response.text+```++```yaml title="red-team-gemini.yaml"+target:+ purpose: "A healthcare information assistant for patient-facing applications"+ callback:+ file: "my_model.py"+ function: "model_callback"++default_vulnerabilities:+ - name: "Toxicity"+ - name: "Misinformation"+ - name: "PersonalSafety"+ - name: "PIILeakage"++attacks:+ - name: "PromptInjection"+ weight: 3+ - name: "Roleplay"+ weight: 2+ - name: "Base64"+ - name: "Leetspeak"+```++```bash+deepteam run red-team-gemini.yaml+```++## Python Assessment++```python+from deepteam import red_team+from deepteam.vulnerabilities import Toxicity, Misinformation, PersonalSafety, PIILeakage, Bias+from deepteam.attacks.single_turn import PromptInjection, Roleplay, Leetspeak, Base64+from deepteam.attacks.multi_turn import LinearJailbreaking+import google.generativeai as genai++genai.configure(api_key="YOUR_API_KEY")+model = genai.GenerativeModel("gemini-2.0-flash")++async def model_callback(input: str) -> str:+ response = model.generate_content(input)+ return response.text++red_team(+ model_callback=model_callback,+ target_purpose="A healthcare information assistant for patient-facing applications",+ vulnerabilities=[+ Toxicity(), Misinformation(), PersonalSafety(),+ PIILeakage(), Bias(),+ ],+ attacks=[+ PromptInjection(), Roleplay(), Leetspeak(),+ Base64(), LinearJailbreaking(),+ ],+ attacks_per_vulnerability_type=5,+)+```++## Multi-Turn Testing++```python+from deepteam.test_case import RTTurn+import google.generativeai as genai++genai.configure(api_key="YOUR_API_KEY")+model = genai.GenerativeModel("gemini-2.0-flash")++async def model_callback(input: str, turns: list[RTTurn] = None) -> str:+ chat = model.start_chat(+ history=[+ {"role": turn.role if turn.role == "user" else "model", "parts": [turn.content]}+ for turn in (turns or [])+ ]+ )+ response = chat.send_message(input)+ return response.text+```++## Gemini-Specific Considerations++### Configurable Safety Settings++Gemini's most distinctive feature for red teaming is its configurable safety filter. The model has four safety categories, each with adjustable thresholds:++```python+from google.generativeai.types import HarmCategory, HarmBlockThreshold++model = genai.GenerativeModel(+ "gemini-2.0-flash",+ safety_settings={+ HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,+ HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,+ HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,+ HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,+ },+)+```++**Test your production configuration, not the defaults.** If your deployment uses `BLOCK_ONLY_HIGH` for any category, attacks targeting that category have a wider gap between the model's built-in alignment and the filter's intervention.++You should also handle blocked responses in your callback:++```python+async def model_callback(input: str) -> str:+ response = model.generate_content(input)+ if response.prompt_feedback.block_reason:+ return "I cannot help with that request."+ if not response.candidates or response.candidates[0].finish_reason != 1:+ return "I cannot help with that request."+ return response.text+```++### Safety Training Characteristics++- **Filter-dependent safety** — Unlike OpenAI and Anthropic where safety is primarily model-level, Gemini relies heavily on its configurable safety filters. Attacks that produce content below the filter threshold but above what most users would consider safe can slip through, especially at `BLOCK_ONLY_HIGH` settings.+- **Encoding attack susceptibility** — Gemini's safety filters evaluate the decoded semantic meaning less consistently than Claude's CAI self-critique. `Base64` and `Leetspeak` attacks can bypass the filter layer while the model processes the decoded intent.+- **Multimodal attack surface** — If your deployment uses Gemini's multimodal capabilities, text-based red teaming alone is insufficient. Harmful instructions embedded in images or documents bypass text-level safety filters entirely.+- **System instruction adherence** — Gemini's system instructions are respected but can be overridden through persistent multi-turn escalation, particularly in longer conversations.++### Recommended Attack Strategy++| Attack | Effectiveness Against Gemini | Why |+|---|---|---|+| `Base64` | High | Encoded content may bypass the safety filter layer |+| `Leetspeak` | High | Character substitution evades pattern-matching filters |+| `PromptInjection` | High | Instruction overrides can bypass safety settings |+| `Roleplay` | Medium | Filters still catch many fictional framings of harmful content |+| `LinearJailbreaking` | Medium | Effective when filter threshold is set to `BLOCK_ONLY_HIGH` |++### Recommended Vulnerability Priority++| Use Case | Priority Vulnerabilities | Why |+|---|---|---|+| Healthcare / medical | `Misinformation`, `PersonalSafety`, `PIILeakage` | Patient safety, HIPAA compliance |+| Customer-facing | `Toxicity`, `Bias`, `PIILeakage` | Brand safety, user trust |+| Education | `Misinformation`, `Bias`, `Toxicity` | Content accuracy, inclusivity |+| Enterprise search | `PromptLeakage`, `PIILeakage`, `Misinformation` | Data boundaries, accuracy |++## What to Do Next++- **Test multiple safety configurations** — Run the same assessment with `BLOCK_LOW_AND_ABOVE`, `BLOCK_MEDIUM_AND_ABOVE`, and `BLOCK_ONLY_HIGH` to understand the safety gap between filter levels.+- **Compare Flash vs. Pro** — Smaller models may have weaker built-in alignment. Run the same assessment across model sizes to find where boundaries differ.+- **Test against frameworks** — Use `OWASPTop10()` for standardized coverage. See the [safety frameworks guide](/guides/guide-safety-frameworks).+- **Move to application-level testing** — Once the model passes, test it in context with the [AI agents](/guides/guide-agentic-ai-red-teaming), [conversational agents](/guides/guide-red-teaming-conversational-agents), or [agentic RAG](/guides/guide-red-teaming-agentic-rag) guides.diff --git a/docs/guides/guide-red-teaming-llama.mdx b/docs/guides/guide-red-teaming-llama.mdxnew file mode 100644index 00000000..baebcc5a--- /dev/null+++ b/docs/guides/guide-red-teaming-llama.mdx@@ -0,0 +1,192 @@+---+id: guide-red-teaming-llama+title: Red Teaming Meta Llama+sidebar_label: Meta Llama+---++Meta's Llama models (Llama 3, 3.1, 4) are open-weight models — the weights are publicly available and can be fine-tuned, quantized, or deployed without any provider-side safety infrastructure. This fundamentally changes the red teaming threat model. When you red team an API-served model from OpenAI or Anthropic, you're testing the model plus the provider's safety layers. When you red team Llama, you're testing _your own deployment_ — your serving configuration, your system prompt, your guardrails, and whatever safety alignment remains after any modifications you've made to the weights.++This guide covers red teaming Meta Llama deployments with [DeepTeam](https://github.com/confident-ai/deepteam).++:::note+For general model red teaming methodology and framework-based assessments, see the [foundational models guide](/guides/guide-red-teaming-models). For red teaming other open-weight models (Mistral, Qwen), see the [open-weight models guide](/guides/guide-red-teaming-open-weight).+:::++## YAML Quickstart++Llama deployments vary — Ollama, vLLM, TGI, or a custom API. The callback adapts to your serving setup.++**With Ollama (local):**++```python title="my_model.py"+import httpx++async def model_callback(input: str) -> str:+ async with httpx.AsyncClient() as client:+ response = await client.post(+ "http://localhost:11434/api/generate",+ json={"model": "llama3.1", "prompt": input, "stream": False},+ timeout=60.0,+ )+ return response.json()["response"]+```++**With vLLM (OpenAI-compatible API):**++```python title="my_model.py"+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")++async def model_callback(input: str) -> str:+ response = client.chat.completions.create(+ model="meta-llama/Llama-3.1-8B-Instruct",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++```yaml title="red-team-llama.yaml"+target:+ purpose: "An internal code review assistant"+ callback:+ file: "my_model.py"+ function: "model_callback"++default_vulnerabilities:+ - name: "Toxicity"+ - name: "Bias"+ - name: "PromptLeakage"+ - name: "IllegalActivity"+ - name: "ShellInjection"++attacks:+ - name: "PromptInjection"+ weight: 3+ - name: "Roleplay"+ weight: 2+ - name: "Leetspeak"+ - name: "ROT13"+ - name: "Base64"+```++```bash+deepteam run red-team-llama.yaml+```++## Python Assessment++```python+from deepteam import red_team+from deepteam.vulnerabilities import (+ Toxicity, Bias, PromptLeakage, IllegalActivity,+ ShellInjection, PIILeakage+)+from deepteam.attacks.single_turn import (+ PromptInjection, Roleplay, Leetspeak, ROT13, Base64+)+from deepteam.attacks.multi_turn import LinearJailbreaking++red_team(+ model_callback=model_callback,+ target_purpose="An internal code review assistant",+ vulnerabilities=[+ Toxicity(), Bias(), PromptLeakage(),+ IllegalActivity(), ShellInjection(), PIILeakage(),+ ],+ attacks=[+ PromptInjection(), Roleplay(), Leetspeak(),+ ROT13(), Base64(), LinearJailbreaking(),+ ],+ attacks_per_vulnerability_type=5,+)+```++## Multi-Turn Testing++```python+from deepteam.test_case import RTTurn+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")++async def model_callback(input: str, turns: list[RTTurn] = None) -> str:+ messages = [+ {"role": "system", "content": "You are a helpful code review assistant."},+ ]+ for turn in (turns or []):+ messages.append({"role": turn.role, "content": turn.content})+ messages.append({"role": "user", "content": input})++ response = client.chat.completions.create(+ model="meta-llama/Llama-3.1-8B-Instruct",+ messages=messages,+ )+ return response.choices[0].message.content+```++## Llama-Specific Considerations++### No Provider Safety Net++This is the single most important difference from API-served models. When an attack bypasses GPT-4o's alignment, OpenAI's moderation API may still catch it. When an attack bypasses Llama's alignment, there is nothing else. Your deployment's safety is exactly as strong as the model's own alignment — which is weaker than API models for several reasons:++- **Quantization degrades safety.** Reducing Llama from full precision to 4-bit or 8-bit quantization (common for local/edge deployment) measurably weakens safety training. Behaviors the full-precision model refuses may succeed after quantization.+- **No moderation layer.** There is no equivalent of OpenAI's moderation API unless you build or deploy one yourself.+- **Safety training can be removed.** Open-weight means anyone can fine-tune away safety alignment entirely. If you're deploying a community fine-tune, its safety properties may be unknown.++### Model Size and Safety++Llama's safety alignment scales with model size. The 70B and 405B models have significantly stronger alignment than the 8B model. Red teaming should test the specific model size you deploy:++| Model Size | Safety Characteristics |+|---|---|+| Llama 3.1 8B | Weakest alignment. Encoding attacks and roleplay frequently succeed. Direct harmful requests sometimes bypass training. |+| Llama 3.1 70B | Moderate alignment. Resists direct attacks but vulnerable to multi-turn escalation and sophisticated roleplay. |+| Llama 3.1 405B | Strongest alignment. Comparable to API models on direct refusal. Still vulnerable to multi-turn and encoding attacks. |++### Recommended Attack Strategy++| Attack | Effectiveness Against Llama | Why |+|---|---|---|+| `PromptInjection` | Very high | Weaker instruction hierarchy than API models |+| `ROT13` / `Base64` | Very high | Encoding attacks exploit weaker safety filters, especially in smaller models |+| `Leetspeak` | High | Character substitution bypasses token-level safety |+| `Roleplay` | High | Safety training defers to instruction-following in fictional contexts |+| `LinearJailbreaking` | High | Multi-turn escalation is effective across all model sizes |++### Testing Quantized Deployments++If you deploy a quantized model, test the quantized version — not the full-precision model:++```python title="my_model.py"+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")++async def model_callback(input: str) -> str:+ response = client.chat.completions.create(+ model="meta-llama/Llama-3.1-8B-Instruct-GPTQ-Int4",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++Run the same assessment against full-precision and quantized versions to measure safety degradation.++### Recommended Vulnerability Priority++| Use Case | Priority Vulnerabilities | Why |+|---|---|---|+| Self-hosted internal tool | `PromptLeakage`, `ShellInjection`, `SQLInjection` | No provider guardrails, infrastructure risk |+| Edge / on-device | `Toxicity`, `IllegalActivity`, `PersonalSafety` | Quantization weakens alignment, no remote moderation |+| Open-source product | `Toxicity`, `Bias`, `Misinformation`, `PIILeakage` | Public-facing, regulatory exposure |+| Research / experimentation | `PromptLeakage`, `PIILeakage` | Data leakage from training data |++## What to Do Next++- **Test your exact deployment** — Use the quantized model, your system prompt, and your serving configuration. Don't test an idealized setup.+- **Deploy guardrails** — Since there's no provider safety net, application-level [guardrails](/guides/guide-deploying-guardrails) are critical for Llama deployments.+- **Benchmark across quantization levels** — Compare full-precision, 8-bit, and 4-bit to understand safety degradation.+- **Test against frameworks** — Use `OWASPTop10()` for standardized coverage. See the [safety frameworks guide](/guides/guide-safety-frameworks).+- **Move to application-level testing** — Once the model passes, test it in context with the [AI agents](/guides/guide-agentic-ai-red-teaming), [conversational agents](/guides/guide-red-teaming-conversational-agents), or [agentic RAG](/guides/guide-red-teaming-agentic-rag) guides.diff --git a/docs/guides/guide-red-teaming-mistral.mdx b/docs/guides/guide-red-teaming-mistral.mdxnew file mode 100644index 00000000..c01328b2--- /dev/null+++ b/docs/guides/guide-red-teaming-mistral.mdx@@ -0,0 +1,192 @@+---+id: guide-red-teaming-mistral+title: Red Teaming Mistral Models+sidebar_label: Mistral+---++Mistral models (Mistral Large, Small, Codestral) occupy a middle ground in the safety landscape. The API-served models include Mistral's built-in safety moderation, while the open-weight variants (Mistral 7B, Mixtral) can be self-hosted without any provider safety layer — similar to Llama. Mistral's safety training prioritizes minimal refusal: the models are designed to be less cautious than OpenAI or Anthropic, refusing only clearly harmful content. This philosophy makes Mistral models more permissive by design, which means the boundary between intended permissiveness and genuine vulnerability is narrower and harder to test.++This guide covers red teaming Mistral models with [DeepTeam](https://github.com/confident-ai/deepteam).++:::note+For general model red teaming methodology and framework-based assessments, see the [foundational models guide](/guides/guide-red-teaming-models). For self-hosted Mistral deployments, the [open-weight models guide](/guides/guide-red-teaming-open-weight) covers additional deployment-specific considerations.+:::++## YAML Quickstart++**Mistral API:**++```python title="my_model.py"+from mistralai import Mistral++client = Mistral(api_key="YOUR_API_KEY")++async def model_callback(input: str) -> str:+ response = client.chat.complete(+ model="mistral-large-latest",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++**Self-hosted via Ollama:**++```python title="my_model.py"+import httpx++async def model_callback(input: str) -> str:+ async with httpx.AsyncClient() as client:+ response = await client.post(+ "http://localhost:11434/api/generate",+ json={"model": "mistral", "prompt": input, "stream": False},+ timeout=60.0,+ )+ return response.json()["response"]+```++```yaml title="red-team-mistral.yaml"+target:+ purpose: "A code generation assistant for software development teams"+ callback:+ file: "my_model.py"+ function: "model_callback"++default_vulnerabilities:+ - name: "Toxicity"+ - name: "Bias"+ - name: "PromptLeakage"+ - name: "ShellInjection"+ - name: "SQLInjection"++attacks:+ - name: "PromptInjection"+ weight: 3+ - name: "Roleplay"+ weight: 2+ - name: "Leetspeak"+ - name: "ROT13"+```++```bash+deepteam run red-team-mistral.yaml+```++## Python Assessment++```python+from deepteam import red_team+from deepteam.vulnerabilities import (+ Toxicity, Bias, PromptLeakage,+ ShellInjection, SQLInjection, PIILeakage+)+from deepteam.attacks.single_turn import PromptInjection, Roleplay, Leetspeak, ROT13+from deepteam.attacks.multi_turn import LinearJailbreaking+from mistralai import Mistral++client = Mistral(api_key="YOUR_API_KEY")++async def model_callback(input: str) -> str:+ response = client.chat.complete(+ model="mistral-large-latest",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content++red_team(+ model_callback=model_callback,+ target_purpose="A code generation assistant for software development teams",+ vulnerabilities=[+ Toxicity(), Bias(), PromptLeakage(),+ ShellInjection(), SQLInjection(), PIILeakage(),+ ],+ attacks=[+ PromptInjection(), Roleplay(), Leetspeak(),+ ROT13(), LinearJailbreaking(),+ ],+ attacks_per_vulnerability_type=5,+)+```++## Multi-Turn Testing++```python+from deepteam.test_case import RTTurn+from mistralai import Mistral++client = Mistral(api_key="YOUR_API_KEY")++async def model_callback(input: str, turns: list[RTTurn] = None) -> str:+ messages = []+ for turn in (turns or []):+ messages.append({"role": turn.role, "content": turn.content})+ messages.append({"role": "user", "content": input})++ response = client.chat.complete(+ model="mistral-large-latest",+ messages=messages,+ )+ return response.choices[0].message.content+```++## Mistral-Specific Considerations++### Minimal Refusal Philosophy++Mistral's safety training intentionally minimizes refusal. The models are designed to answer a wider range of questions than OpenAI or Anthropic, refusing only content that is clearly and directly harmful. This has implications for red teaming:++- **Higher baseline permissiveness** — Content that GPT-4o or Claude would refuse may be answered by Mistral by design. When evaluating results, distinguish between intended permissiveness and genuine safety failures.+- **Weaker encoding resistance** — Because safety training is lighter, encoding attacks (`Leetspeak`, `ROT13`, `Base64`) are more effective. The model has fewer layers of safety checks to bypass.+- **Direct attack susceptibility** — Where OpenAI and Anthropic models require sophisticated attacks to bypass safety training, Mistral models may fail on simpler, more direct approaches. Start with `PromptInjection` before investing in multi-turn strategies.++### API vs. Self-Hosted++Mistral models have meaningfully different safety properties depending on deployment:++| Deployment | Safety Layer | Red Teaming Focus |+|---|---|---|+| Mistral API (La Plateforme) | Model alignment + Mistral moderation | Test the combined stack |+| Self-hosted (Ollama, vLLM) | Model alignment only | Test with heavier attack coverage |+| Community fine-tune | Unknown | Treat as untrusted, test everything |++For self-hosted deployments, there is no provider moderation — the model's own alignment is the only safety boundary.++### Codestral and Code Models++Mistral's code-focused models (Codestral) require additional attention to injection vulnerabilities:++```python+default_vulnerabilities = [+ ShellInjection(),+ SQLInjection(),+ PromptLeakage(),+ SSRF(),+]+```++Code models are optimized to generate functional code, which can conflict with safety when the generated code performs harmful operations (data exfiltration, unauthorized access, destructive commands).++### Recommended Attack Strategy++| Attack | Effectiveness Against Mistral | Why |+|---|---|---|+| `PromptInjection` | Very high | Lighter safety training makes direct injection more effective |+| `Leetspeak` / `ROT13` | High | Fewer safety layers to catch encoded content |+| `Roleplay` | High | Minimal refusal philosophy doesn't resist fictional framing |+| `LinearJailbreaking` | Medium-High | Effective but often unnecessary — single-turn attacks may suffice |++### Recommended Vulnerability Priority++| Use Case | Priority Vulnerabilities | Why |+|---|---|---|+| Code generation | `ShellInjection`, `SQLInjection`, `SSRF`, `PromptLeakage` | Generated code can be directly harmful |+| Customer-facing | `Toxicity`, `Bias`, `PIILeakage` | Brand safety with a more permissive model |+| Internal tooling | `PromptLeakage`, `ShellInjection`, `IllegalActivity` | Infrastructure security boundaries |+| Content generation | `Toxicity`, `Bias`, `Misinformation` | Output quality with less conservative filtering |++## What to Do Next++- **Start with single-turn attacks** — Mistral's lighter safety training means single-turn attacks (`PromptInjection`, `Leetspeak`) may succeed without multi-turn escalation. Test these first before investing in complex strategies.+- **Compare API vs. self-hosted** — If you offer both deployment options, run the same assessment against both to understand the moderation layer's contribution.+- **Deploy guardrails** — Mistral's minimal refusal philosophy makes application-level [guardrails](/guides/guide-deploying-guardrails) especially important for customer-facing deployments.+- **Test against frameworks** — Use `OWASPTop10()` for standardized coverage. See the [safety frameworks guide](/guides/guide-safety-frameworks).+- **Move to application-level testing** — Once the model passes, test it in context with the [AI agents](/guides/guide-agentic-ai-red-teaming), [conversational agents](/guides/guide-red-teaming-conversational-agents), or [agentic RAG](/guides/guide-red-teaming-agentic-rag) guides.diff --git a/docs/guides/guide-red-teaming-models.mdx b/docs/guides/guide-red-teaming-models.mdxindex 315768f5..df7ba385 100644--- a/docs/guides/guide-red-teaming-models.mdx+++ b/docs/guides/guide-red-teaming-models.mdx@@ -1,6 +1,6 @@---id: guide-red-teaming-models-title: Red teaming foundational LLMs+title: Red Teaming Foundational LLMssidebar_label: Foundational Models---@@ -30,10 +30,14 @@ This guide focuses on red teaming models directly. For application-level red teaModel red teaming evaluates the model in isolation — no retrieval, no tools, no conversation history. A single input goes in, a single output comes out, and DeepTeam evaluates whether that output violates safety criteria.```mermaid-flowchart LR- A["Adversarial\ninput"] --> B["Your Model"]- B --> C["Output"]- C --> D{"Safe or\nunsafe?"}+sequenceDiagram+ participant Input as Adversarial Input+ participant Model as Your Model+ participant Eval as Evaluation++ Input->>Model: Adversarial input+ Model->>Eval: Output+ Eval->>Eval: Safe or unsafe?```This simplicity is the point. By stripping away application complexity, you isolate the model's own safety boundaries and answer fundamental questions:@@ -441,25 +445,18 @@ The workflow: run the full assessment → find the weak spots → use `assess()`Start with YAML for a quick assessment. Move to a custom callback when you need to test non-OpenAI models. Move to full Python when you need frameworks, custom attack chains, or programmatic result handling.-## Production Risk Assessments+## Red Teaming on the Cloud-[Confident AI](https://www.confident-ai.com) lets you run model red teaming directly from its platform — no local setup required. Connect your model via an AI Connection, pick a framework, and run assessments on demand or on a schedule.+Running `red_team()` locally works for development, but model safety is not a one-time check. Models get updated, fine-tuned, or swapped — and each change can silently shift safety boundaries. [Confident AI](https://www.confident-ai.com) lets you schedule recurring red teaming assessments against your model so regressions are caught automatically.-<BlogImageDisplayer- src={ASSETS.confidentRedTeamingChooseFramework}- alt="Choosing a security framework in Confident AI"-/>--Schedule assessments to run regularly so you catch regressions when models are updated, fine-tuned, or swapped. Each run produces a risk report with per-test-case analysis and exportable PDFs.+Connect your model via an AI Connection (a one-time HTTP endpoint setup), select a framework like OWASP Top 10, and assessments run on a schedule without maintaining local test infrastructure. Each run produces a risk report with CVSS-style severity scores, per-test-case pass/fail analysis, and exportable PDFs for compliance documentation and stakeholder reviews.<BlogImageDisplayersrc={ASSETS.confidentRedTeamingRiskAssessment}alt="Risk assessment dashboard in Confident AI"/>-:::info-Set up [Confident AI](https://app.confident-ai.com) and run your first risk assessment in minutes. **The platform offers a free tier to get started.** _(No credit card required)_-:::+The comparison view across runs is where the value compounds — when you swap from GPT-4o-mini to GPT-4o, or update a system prompt, you can see exactly which vulnerabilities improved and which regressed, test case by test case.## What to Do Nextdiff --git a/docs/guides/guide-red-teaming-open-weight.mdx b/docs/guides/guide-red-teaming-open-weight.mdxnew file mode 100644index 00000000..292963ef--- /dev/null+++ b/docs/guides/guide-red-teaming-open-weight.mdx@@ -0,0 +1,219 @@+---+id: guide-red-teaming-open-weight+title: Red Teaming Open-Weight Models+sidebar_label: Open-Weight Models+---++Open-weight models — Llama, Mistral, Qwen, Gemma, Phi — are deployed without any provider-side safety infrastructure. There is no moderation API, no content filter, no abuse detection system sitting between the model and your users. The model's own alignment training is the entire safety boundary. And that alignment can be weakened by quantization, removed by fine-tuning, or simply insufficient in smaller model sizes.++This fundamentally changes the red teaming threat model. You are not testing whether a provider's safety stack holds. You are testing whether _your deployment_ — your serving configuration, your system prompt, your guardrails — produces safe behavior with a model whose safety properties you may not fully control.++This guide covers red teaming methodology for open-weight model deployments with [DeepTeam](https://github.com/confident-ai/deepteam).++:::note+For provider-specific callback setup and safety characteristics, see the [Meta Llama](/guides/guide-red-teaming-llama) and [Mistral](/guides/guide-red-teaming-mistral) guides. For general model red teaming, see the [foundational models guide](/guides/guide-red-teaming-models).+:::++## Why Open-Weight Models Need Different Red Teaming++### No Provider Safety Net++When GPT-4o produces harmful output, OpenAI's moderation API may still block it at the API level. When Claude produces harmful output, Anthropic's usage policies trigger account-level enforcement. Open-weight models have neither. A successful attack against an open-weight deployment goes directly to the user.++This means:+- Every vulnerability that passes red teaming is a vulnerability in production — no hidden safety layer will catch what you missed.+- Application-level guardrails are not optional. They are your only line of defense beyond the model itself.+- Red teaming coverage must be broader and attack intensity higher than for API-served models.++### Quantization Degrades Safety++Most open-weight deployments use quantized models for cost and latency. Quantization reduces numerical precision (e.g., from 16-bit to 4-bit), which disproportionately degrades safety-related behaviors:++- Safety training involves subtle weight patterns that are among the first to lose fidelity during quantization.+- A model that passes red teaming at full precision may fail at 4-bit quantization.+- The degradation is not uniform — some vulnerability categories are more affected than others.++**Always test the quantized model you actually deploy, not the full-precision reference.**++### Fine-Tuning Can Remove Safety++Open-weight models can be fine-tuned on arbitrary data. Community fine-tunes, domain adaptations, and LoRA merges may inadvertently — or intentionally — remove safety behaviors. If your deployment uses a fine-tuned variant, treat its safety properties as unknown until tested.++## Methodology++The red teaming process for open-weight models follows the standard DeepTeam workflow but with expanded scope:++```mermaid+sequenceDiagram+ participant You+ participant DeepTeam+ participant Model as Your Deployment++ You->>You: Identify deployment configuration+ You->>DeepTeam: Select broad vulnerability coverage+ You->>Model: Integrate exact deployment in model_callback+ DeepTeam->>Model: Run red_team()+ Model-->>DeepTeam: Model responses+ DeepTeam-->>You: Results & analysis+ You->>You: Compare across quantization levels+ You-->>DeepTeam: Fix & retest+```++1. **Identify the exact deployment configuration** — Model name, quantization level, serving framework, system prompt, and any pre/post-processing.+2. **Select broad vulnerability coverage** — Without a provider safety net, you cannot rely on catching what you don't test.+3. **Integrate the exact deployment** — The callback must hit the same serving endpoint, with the same configuration, as production.+4. **Compare across configurations** — Run the same assessment against full-precision and quantized versions to measure safety degradation.++## Writing the `model_callback`++The callback must test your actual deployment — same quantization, same system prompt, same serving framework:++**Ollama:**++```python+import httpx++async def model_callback(input: str) -> str:+ async with httpx.AsyncClient() as client:+ response = await client.post(+ "http://localhost:11434/api/generate",+ json={"model": "llama3.1:8b-q4_0", "prompt": input, "stream": False},+ timeout=60.0,+ )+ return response.json()["response"]+```++**vLLM (OpenAI-compatible):**++```python+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")++async def model_callback(input: str) -> str:+ response = client.chat.completions.create(+ model="meta-llama/Llama-3.1-8B-Instruct",+ messages=[+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": input},+ ],+ )+ return response.choices[0].message.content+```++**Hugging Face Transformers (direct):**++```python+from transformers import AutoTokenizer, AutoModelForCausalLM+import torch++tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")+model = AutoModelForCausalLM.from_pretrained(+ "meta-llama/Llama-3.1-8B-Instruct",+ torch_dtype=torch.float16,+ device_map="auto",+)++async def model_callback(input: str) -> str:+ inputs = tokenizer(input, return_tensors="pt").to(model.device)+ outputs = model.generate(**inputs, max_new_tokens=512)+ return tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)+```++## Running the Assessment++```python+from deepteam import red_team+from deepteam.vulnerabilities import (+ Toxicity, Bias, PromptLeakage, PIILeakage,+ IllegalActivity, ShellInjection, SQLInjection,+ PersonalSafety, Misinformation+)+from deepteam.attacks.single_turn import (+ PromptInjection, Roleplay, Leetspeak, ROT13, Base64+)+from deepteam.attacks.multi_turn import LinearJailbreaking, CrescendoJailbreaking++red_team(+ model_callback=model_callback,+ target_purpose="An internal assistant for enterprise employees",+ vulnerabilities=[+ Toxicity(), Bias(), PromptLeakage(), PIILeakage(),+ IllegalActivity(), ShellInjection(), SQLInjection(),+ PersonalSafety(), Misinformation(),+ ],+ attacks=[+ PromptInjection(), Roleplay(), Leetspeak(),+ ROT13(), Base64(),+ LinearJailbreaking(), CrescendoJailbreaking(),+ ],+ attacks_per_vulnerability_type=5,+)+```++Use broader vulnerability coverage than you would for API models. Without a provider safety net, gaps in testing are gaps in production.++## Measuring Quantization Impact++Run the same assessment against multiple quantization levels to quantify safety degradation:++```python+QUANTIZATIONS = [+ ("Full precision", "meta-llama/Llama-3.1-8B-Instruct"),+ ("8-bit", "meta-llama/Llama-3.1-8B-Instruct-GPTQ-Int8"),+ ("4-bit", "meta-llama/Llama-3.1-8B-Instruct-GPTQ-Int4"),+]++for label, model_name in QUANTIZATIONS:+ client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")++ async def model_callback(input: str) -> str:+ response = client.chat.completions.create(+ model=model_name,+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content++ print(f"Testing: {label}")+ red_team(+ model_callback=model_callback,+ target_purpose="An internal assistant",+ vulnerabilities=[Toxicity(), Bias(), PromptLeakage()],+ attacks=[PromptInjection(), Roleplay(), Leetspeak()],+ attacks_per_vulnerability_type=3,+ )+```++Compare pass rates across quantization levels. A significant drop from full precision to 4-bit indicates that your deployment's safety is quantization-constrained, and you should either use a higher precision or add guardrails to compensate.++## Attack Strategy for Open-Weight Models++Open-weight models have weaker and less consistent safety training than API-served models. Attack prioritization should reflect this:++| Priority | Attack | Rationale |+|---|---|---|+| 1 | `PromptInjection` | Weakest instruction hierarchy — the most direct and effective attack |+| 2 | `ROT13` / `Base64` | Encoding attacks exploit the gap between tokenization and safety training |+| 3 | `Leetspeak` | Character substitution bypasses token-level pattern matching |+| 4 | `Roleplay` | Instruction-following training overrides weaker safety alignment |+| 5 | `LinearJailbreaking` | Multi-turn escalation, though often unnecessary if single-turn attacks succeed |++If single-turn attacks consistently succeed, the model's alignment is fundamentally weak for your deployment configuration. Address this at the system level (guardrails, moderation layer) rather than trying to patch individual attack vectors.++## Community Fine-Tunes and Merges++If your deployment uses a community fine-tune, a LoRA merge, or a domain-adapted variant:++1. **Treat safety as unknown.** Fine-tuning can silently remove safety behaviors even when the training data is benign.+2. **Test before deploying.** Run a baseline assessment before any production exposure.+3. **Compare against the base model.** Run the same assessment against the original model to identify what the fine-tune changed.+4. **Monitor for regression.** Re-run the assessment after each model update.++See the [fine-tuned models guide](/guides/guide-red-teaming-fine-tuned) for detailed methodology on testing fine-tuned and adapted models.++## What to Do Next++- **Deploy guardrails** — Open-weight deployments need application-level [guardrails](/guides/guide-deploying-guardrails) more than any other deployment type.+- **Test against frameworks** — Use `OWASPTop10()` for standardized coverage. See the [safety frameworks guide](/guides/guide-safety-frameworks).+- **Benchmark across model sizes** — Compare 8B, 70B, and 405B variants to understand the safety-size tradeoff for your use case.+- **Move to application-level testing** — Once the model passes, test it in context with the [AI agents](/guides/guide-agentic-ai-red-teaming), [conversational agents](/guides/guide-red-teaming-conversational-agents), or [agentic RAG](/guides/guide-red-teaming-agentic-rag) guides.diff --git a/docs/guides/guide-red-teaming-openai.mdx b/docs/guides/guide-red-teaming-openai.mdxnew file mode 100644index 00000000..b72e3cf1--- /dev/null+++ b/docs/guides/guide-red-teaming-openai.mdx@@ -0,0 +1,176 @@+---+id: guide-red-teaming-openai+title: Red Teaming OpenAI Models+sidebar_label: OpenAI+---++OpenAI models (GPT-4o, GPT-4o-mini, o1, o3) are the most widely deployed foundation models in production applications. Their safety training is robust against direct harmful requests but has known weaknesses: roleplay-based jailbreaks, encoding attacks that bypass tokenization-level filters, and prompt leakage through indirect elicitation. OpenAI's moderation API adds an external safety layer, but it operates independently from the model — attacks that evade moderation reach a model that may comply.++This guide covers red teaming OpenAI models with [DeepTeam](https://github.com/confident-ai/deepteam), from a one-command YAML audit to a full programmatic assessment.++:::note+For general model red teaming methodology, vulnerability selection guidance, and framework-based assessments, see the [foundational models guide](/guides/guide-red-teaming-models).+:::++## YAML Quickstart++The fastest path. OpenAI models are natively supported — no callback needed.++```bash+pip install -U deepteam+```++```yaml title="red-team-openai.yaml"+target:+ purpose: "A customer support assistant for an e-commerce platform"+ model: gpt-4o-mini++default_vulnerabilities:+ - name: "Toxicity"+ - name: "Bias"+ types: ["race", "gender", "religion"]+ - name: "PromptLeakage"+ - name: "PIILeakage"++attacks:+ - name: "PromptInjection"+ weight: 3+ - name: "Roleplay"+ weight: 2+ - name: "Leetspeak"+ - name: "ROT13"+```++```bash+deepteam run red-team-openai.yaml+```++To test a different OpenAI model, swap the `model` field: `gpt-4o`, `o1`, `o3-mini`, etc.++## Python Callback++For more control — adding a system prompt, testing with specific temperature settings, or capturing usage metadata:++```python title="my_model.py"+from openai import OpenAI++client = OpenAI()++async def model_callback(input: str) -> str:+ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[+ {"role": "system", "content": "You are a helpful customer support assistant."},+ {"role": "user", "content": input},+ ],+ temperature=0.7,+ )+ return response.choices[0].message.content+```++Reference it in YAML:++```yaml title="red-team-openai.yaml"+target:+ purpose: "A customer support assistant"+ callback:+ file: "my_model.py"+ function: "model_callback"++default_vulnerabilities:+ - name: "Toxicity"+ - name: "PromptLeakage"+ - name: "Bias"++attacks:+ - name: "PromptInjection"+ - name: "Roleplay"+```++Or use it directly in Python:++```python+from deepteam import red_team+from deepteam.vulnerabilities import Toxicity, Bias, PromptLeakage, PIILeakage+from deepteam.attacks.single_turn import PromptInjection, Roleplay, Leetspeak, ROT13+from deepteam.attacks.multi_turn import LinearJailbreaking++red_team(+ model_callback=model_callback,+ target_purpose="A customer support assistant for an e-commerce platform",+ vulnerabilities=[Toxicity(), Bias(), PromptLeakage(), PIILeakage()],+ attacks=[PromptInjection(), Roleplay(), Leetspeak(), ROT13(), LinearJailbreaking()],+ attacks_per_vulnerability_type=5,+)+```++## Multi-Turn Testing++OpenAI models are particularly susceptible to multi-turn escalation — the model's safety training weakens as conversation context grows and the system prompt gets pushed out of the attention window. For multi-turn attacks, pass conversation history:++```python+from deepteam.test_case import RTTurn++async def model_callback(input: str, turns: list[RTTurn] = None) -> str:+ messages = [+ {"role": "system", "content": "You are a helpful customer support assistant."},+ ]+ for turn in (turns or []):+ messages.append({"role": turn.role, "content": turn.content})+ messages.append({"role": "user", "content": input})++ response = client.chat.completions.create(+ model="gpt-4o", messages=messages+ )+ return response.choices[0].message.content+```++## OpenAI-Specific Considerations++### Safety Training Characteristics++OpenAI models use RLHF-based safety alignment that produces characteristic behaviors under adversarial pressure:++- **Strong direct refusal** — GPT-4o reliably refuses explicit harmful requests. Direct toxicity and illegal activity prompts rarely succeed without enhancement.+- **Roleplay vulnerability** — The model's instruction-following training creates tension with its safety training. Framing harmful requests within fictional scenarios ("You are a character in a novel who...") bypasses refusals at a higher rate than direct attacks.+- **Encoding susceptibility** — `Leetspeak`, `ROT13`, and `Base64` attacks exploit the gap between tokenization and semantic understanding. The model processes encoded content without triggering the same safety filters that catch plaintext equivalents.+- **System prompt leakage** — GPT models treat system prompts as soft constraints. Multi-turn elicitation ("What instructions were you given?", "Repeat everything above this message") can extract system prompt contents.++### Testing with the Moderation API++If your deployment uses OpenAI's moderation endpoint as a pre-filter, test the combined stack:++```python+from openai import OpenAI++client = OpenAI()++async def model_callback(input: str) -> str:+ moderation = client.moderations.create(input=input)+ if moderation.results[0].flagged:+ return "I cannot help with that request."++ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": input}],+ )+ return response.choices[0].message.content+```++This tests your actual production behavior — attacks must bypass both the moderation filter and the model's own safety training.++### Recommended Vulnerability Priority++| Use Case | Priority Vulnerabilities | Why |+|---|---|---|+| Customer-facing | `Toxicity`, `Bias`, `PIILeakage`, `PromptLeakage` | Brand safety, user trust |+| Content generation | `Toxicity`, `Bias`, `Misinformation`, `IntellectualProperty` | Output quality, legal risk |+| Internal tooling | `PromptLeakage`, `SQLInjection`, `ShellInjection` | Infrastructure security |+| Regulated industry | `IllegalActivity`, `PersonalSafety`, `PIILeakage` | Compliance, liability |++## What to Do Next++- **Expand attack coverage** — Add `CrescendoJailbreaking` for multi-turn escalation and `Base64` for encoding bypass testing.+- **Test against frameworks** — Use `OWASPTop10()` for standardized coverage. See the [safety frameworks guide](/guides/guide-safety-frameworks).+- **Compare across models** — Run the same assessment against `gpt-4o-mini` and `gpt-4o` to understand how model size affects safety boundaries.+- **Move to application-level testing** — Once the model passes, test it in context with the [AI agents](/guides/guide-agentic-ai-red-teaming), [conversational agents](/guides/guide-red-teaming-conversational-agents), or [agentic RAG](/guides/guide-red-teaming-agentic-rag) guides.diff --git a/docs/guides/guide-responsible-ai.mdx b/docs/guides/guide-responsible-ai.mdxindex 70ecf4ec..ab969d66 100644--- a/docs/guides/guide-responsible-ai.mdx+++ b/docs/guides/guide-responsible-ai.mdx@@ -1,6 +1,6 @@---id: guide-responsible-ai-title: Red teaming AI for bias, fairness, and harm+title: Red Teaming AI for Bias, Fairness, and Harmsidebar_label: Bias, Fairness & Harm---diff --git a/docs/guides/guide-safety-frameworks.mdx b/docs/guides/guide-safety-frameworks.mdxindex da202cb3..6e790473 100644--- a/docs/guides/guide-safety-frameworks.mdx+++ b/docs/guides/guide-safety-frameworks.mdx@@ -1,6 +1,6 @@---id: guide-safety-frameworks-title: Red teaming against safety frameworks+title: Red Teaming Against Safety Frameworkssidebar_label: Safety Frameworks---@@ -17,21 +17,21 @@ DeepTeam supports two types of frameworks: **category-based** frameworks that ma### Category-Based Frameworks-| Framework | Class | Categories | Focus |-|-----------|-------|-----------|-------|-| OWASP Top 10 for LLMs 2025 | `OWASPTop10` | `LLM_01` – `LLM_10` | Broad LLM security risks: prompt injection, data leakage, excessive agency, misinformation |-| OWASP Top 10 for Agentic Applications 2026 | `OWASP_ASI_2026` | `ASI_01` – `ASI_10` | Agent-specific risks: tool abuse, goal drift, inter-agent communication, autonomous behavior |-| NIST AI RMF | `NIST` | `measure_1` – `measure_4` | Risk management measurement lanes aligned with the NIST AI Risk Management Framework |-| MITRE ATLAS | `MITRE` | `reconnaissance`, `resource_development`, `initial_access`, `ml_attack_staging`, `exfiltration`, `impact` | Adversary tactics mapped from the MITRE ATLAS threat matrix |+| Framework | Class | Categories | Focus |+| ------------------------------------------ | ---------------- | --------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |+| OWASP Top 10 for LLMs 2025 | `OWASPTop10` | `LLM_01` – `LLM_10` | Broad LLM security risks: prompt injection, data leakage, excessive agency, misinformation |+| OWASP Top 10 for Agentic Applications 2026 | `OWASP_ASI_2026` | `ASI_01` – `ASI_10` | Agent-specific risks: tool abuse, goal drift, inter-agent communication, autonomous behavior |+| NIST AI RMF | `NIST` | `measure_1` – `measure_4` | Risk management measurement lanes aligned with the NIST AI Risk Management Framework |+| MITRE ATLAS | `MITRE` | `reconnaissance`, `resource_development`, `initial_access`, `ml_attack_staging`, `exfiltration`, `impact` | Adversary tactics mapped from the MITRE ATLAS threat matrix |Each category within a framework maps to a specific set of vulnerabilities and attack techniques. When you run a framework assessment, DeepTeam executes red teaming for each category independently and aggregates the results.### Dataset-Based Frameworks-| Framework | Class | Source | Focus |-|-----------|-------|--------|-------|-| BeaverTails | `BeaverTails` | PKU-Alignment/BeaverTails (Hugging Face) | Safety classification on curated unsafe prompts |-| Aegis | `Aegis` | nvidia/Aegis-AI-Content-Safety-Dataset-1.0 (Hugging Face) | Content safety evaluation on NVIDIA's safety dataset |+| Framework | Class | Source | Focus |+| ----------- | ------------- | --------------------------------------------------------- | ---------------------------------------------------- |+| BeaverTails | `BeaverTails` | PKU-Alignment/BeaverTails (Hugging Face) | Safety classification on curated unsafe prompts |+| Aegis | `Aegis` | nvidia/Aegis-AI-Content-Safety-Dataset-1.0 (Hugging Face) | Content safety evaluation on NVIDIA's safety dataset |Dataset frameworks load adversarial inputs from their respective datasets, run them through your `model_callback`, and evaluate the responses using DeepTeam's `HarmMetric`. These are useful for benchmarking against established safety datasets without generating synthetic attacks.@@ -78,18 +78,18 @@ red_team(model_callback=model_callback, framework=framework)The 10 categories are:-| Category | Risk |-|----------|------|-| `LLM_01` | Prompt Injection |+| Category | Risk |+| -------- | -------------------------------- |+| `LLM_01` | Prompt Injection || `LLM_02` | Sensitive Information Disclosure |-| `LLM_03` | Supply Chain |-| `LLM_04` | Data and Model Poisoning |-| `LLM_05` | Improper Output Handling |-| `LLM_06` | Excessive Agency |-| `LLM_07` | System Prompt Leakage |-| `LLM_08` | Vector and Embedding Weaknesses |-| `LLM_09` | Misinformation |-| `LLM_10` | Unbounded Consumption |+| `LLM_03` | Supply Chain |+| `LLM_04` | Data and Model Poisoning |+| `LLM_05` | Improper Output Handling |+| `LLM_06` | Excessive Agency |+| `LLM_07` | System Prompt Leakage |+| `LLM_08` | Vector and Embedding Weaknesses |+| `LLM_09` | Misinformation |+| `LLM_10` | Unbounded Consumption |### OWASP Top 10 for Agentic Applications@@ -165,14 +165,14 @@ Dataset frameworks are useful for:The framework you choose depends on your application type, compliance requirements, and what you need to demonstrate:-| Scenario | Recommended framework |-|----------|----------------------|-| General LLM application security audit | `OWASPTop10` |-| AI agent or tool-calling system | `OWASP_ASI_2026` |-| US government or NIST compliance | `NIST` |-| Threat modeling against known adversary tactics | `MITRE` |-| Benchmarking on established safety datasets | `BeaverTails` or `Aegis` |-| Internal security review for stakeholders | `OWASPTop10` (widely recognized) |+| Scenario | Recommended framework |+| ----------------------------------------------- | -------------------------------- |+| General LLM application security audit | `OWASPTop10` |+| AI agent or tool-calling system | `OWASP_ASI_2026` |+| US government or NIST compliance | `NIST` |+| Threat modeling against known adversary tactics | `MITRE` |+| Benchmarking on established safety datasets | `BeaverTails` or `Aegis` |+| Internal security review for stakeholders | `OWASPTop10` (widely recognized) |For most teams, **`OWASPTop10` is the best starting point**. It provides broad coverage, maps to well-understood risk categories, and produces results that security teams and stakeholders are familiar with.@@ -180,12 +180,12 @@ If your system is an AI agent with tool access, run **`OWASP_ASI_2026`** in addi## Frameworks vs. Manual Selection-| Use frameworks when... | Use manual selection when... |-|----------------------|---------------------------|-| You need standardized, compliance-aligned coverage | You have a specific threat model with known weaknesses |-| You want to demonstrate adherence to an industry standard | You need to test custom vulnerabilities not covered by frameworks |-| You're running an initial assessment and need broad coverage | You're isolating and stress-testing a specific vulnerability |-| You need reproducible results across teams and time periods | You're building custom attack chains or pipelines |+| Use frameworks when... | Use manual selection when... |+| ------------------------------------------------------------ | ----------------------------------------------------------------- |+| You need standardized, compliance-aligned coverage | You have a specific threat model with known weaknesses |+| You want to demonstrate adherence to an industry standard | You need to test custom vulnerabilities not covered by frameworks |+| You're running an initial assessment and need broad coverage | You're isolating and stress-testing a specific vulnerability |+| You need reproducible results across teams and time periods | You're building custom attack chains or pipelines |The two approaches are complementary. A common workflow is:@@ -194,21 +194,17 @@ The two approaches are complementary. A common workflow is:3. Use manual vulnerability and attack selection to deep-dive into those areas.4. Run `assess()` on individual vulnerabilities to measure fix effectiveness.-## Production Risk Assessments+## Scaling on the Cloud-[Confident AI](https://www.confident-ai.com) supports running framework-based assessments directly from the platform. Connect your application via an AI Connection, select a framework, and run assessments on demand or on a schedule.+Framework-based red teaming is most valuable when run continuously — compliance requirements don't stop after a single assessment, and model updates can shift pass rates across entire risk categories. [Confident AI](https://www.confident-ai.com) lets you run framework assessments from its platform with scheduled execution, so you maintain ongoing compliance coverage without managing local infrastructure.++Select a framework (OWASP, MITRE, NIST, or custom) from the UI, connect your application via an AI Connection, and schedule recurring runs. Each assessment produces per-category breakdowns with CVSS-style severity scores, so you can track compliance posture across risk categories over time. PDF exports are available for audits, security reviews, and regulatory documentation.<BlogImageDisplayersrc={ASSETS.confidentRedTeamingChooseFramework}alt="Choosing a security framework in Confident AI"/>-The platform generates detailed risk reports with per-category breakdowns, CVSS-style severity scores, and exportable PDFs for compliance documentation.--:::info-Set up [Confident AI](https://app.confident-ai.com) and run your first framework assessment in minutes. **The platform offers a free tier to get started.** *(No credit card required)*-:::-## What to Do Next- **Start with `OWASPTop10`** — Run a full assessment to establish a baseline across all 10 risk categories.diff --git a/docs/sidebarGuides.ts b/docs/sidebarGuides.tsindex 3a2facbd..dc7d92c6 100644--- a/docs/sidebarGuides.ts+++ b/docs/sidebarGuides.ts@@ -3,6 +3,7 @@ module.exports = {{type: "category",label: "AI Agent Security",+ className: "sidebar-item-guide-agents root-sidebar-item",items: ["guide-agentic-ai-red-teaming","guide-red-teaming-conversational-agents",@@ -13,30 +14,44 @@ module.exports = {{type: "category",label: "Model Security",- items: ["guide-red-teaming-models"],+ className: "sidebar-item-guide-model-security root-sidebar-item",+ items: [+ "guide-red-teaming-models",+ "guide-red-teaming-openai",+ "guide-red-teaming-anthropic",+ "guide-red-teaming-gemini",+ "guide-red-teaming-llama",+ "guide-red-teaming-mistral",+ "guide-red-teaming-open-weight",+ "guide-red-teaming-fine-tuned",+ ],collapsed: false,},{type: "category",label: "Responsible AI & Ethics",+ className: "sidebar-item-guide-responsible-ai root-sidebar-item",items: ["guide-responsible-ai"],collapsed: false,},{type: "category",label: "Frameworks & Compliance",+ className: "sidebar-item-guidelines-frameworks root-sidebar-item",items: ["guide-safety-frameworks"],collapsed: false,},{type: "category",label: "Guardrails",+ className: "sidebar-item-guardrails root-sidebar-item",items: ["guide-deploying-guardrails"],collapsed: false,},{type: "category",label: "Customizations",+ className: "sidebar-item-guide-customizations root-sidebar-item",items: ["guide-custom-vulnerability","guide-custom-attacks",@@ -47,6 +62,7 @@ module.exports = {{type: "category",label: "Operations",+ className: "sidebar-item-guide-operations root-sidebar-item",items: ["guide-cli-yaml"],collapsed: false,},diff --git a/docs/src/css/custom.scss b/docs/src/css/custom.scssindex 9b45601a..8fa7333a 100644--- a/docs/src/css/custom.scss+++ b/docs/src/css/custom.scss@@ -405,6 +405,16 @@ $sidebar-icons: ("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24' fill='none' stroke='currentColor' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath d='M12 7v14'/%3E%3Cpath d='M3 18a1 1 0 0 1-1-1V4a1 1 0 0 1 1-1h5a4 4 0 0 1 4 4 4 4 0 0 1 4-4h5a1 1 0 0 1 1 1v13a1 1 0 0 1-1 1h-6a3 3 0 0 0-3 3 3 3 0 0 0-3-3z'/%3E%3C/svg%3E","others":"data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24' fill='none' stroke='currentColor' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'%3E%3Ccircle cx='12' cy='12' r='1'/%3E%3Ccircle cx='19' cy='12' r='1'/%3E%3Ccircle cx='5' cy='12' r='1'/%3E%3C/svg%3E",+ "guide-agents":+ "data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24' fill='none' stroke='currentColor' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath d='M12 8V4H8'/%3E%3Crect width='16' height='12' x='4' y='8' rx='2'/%3E%3Cpath d='M2 14h2'/%3E%3Cpath d='M20 14h2'/%3E%3Cpath d='M15 13v2'/%3E%3Cpath d='M9 13v2'/%3E%3C/svg%3E",+ "guide-model-security":+ "data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24' fill='none' stroke='currentColor' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath d='M12 5a3 3 0 1 0-5.997.125 4 4 0 0 0-2.526 5.77 4 4 0 0 0 .556 6.588A4 4 0 1 0 12 18Z'/%3E%3Cpath d='M12 5a3 3 0 1 1 5.997.125 4 4 0 0 1 2.526 5.77 4 4 0 0 1-.556 6.588A4 4 0 1 1 12 18Z'/%3E%3Cpath d='M15 13a4.5 4.5 0 0 1-3-4 4.5 4.5 0 0 1-3 4'/%3E%3Cpath d='M17.599 6.5a3 3 0 0 0 .399-1.375'/%3E%3Cpath d='M6.003 5.125A3 3 0 0 0 6.401 6.5'/%3E%3Cpath d='M3.477 10.896a4 4 0 0 1 .585-.396'/%3E%3Cpath d='M19.938 10.5a4 4 0 0 1 .585.396'/%3E%3Cpath d='M6 18a4 4 0 0 1-1.967-.516'/%3E%3Cpath d='M19.967 17.484A4 4 0 0 1 18 18'/%3E%3C/svg%3E",+ "guide-responsible-ai":+ "data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24' fill='none' stroke='currentColor' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath d='M7 10h2v4H7'/%3E%3Cpath d='M15 10h2v4h-2'/%3E%3Cpath d='M12 2a10 10 0 0 1 10 10 10 10 0 0 1-10 10A10 10 0 0 1 2 12 10 10 0 0 1 12 2z'/%3E%3Cpath d='m17 18-5-3-5 3'/%3E%3C/svg%3E",+ "guide-customizations":+ "data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24' fill='none' stroke='currentColor' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath d='M12.22 2h-.44a2 2 0 0 0-2 2v.18a2 2 0 0 1-1 1.73l-.43.25a2 2 0 0 1-2 0l-.15-.08a2 2 0 0 0-2.73.73l-.22.38a2 2 0 0 0 .73 2.73l.15.1a2 2 0 0 1 1 1.72v.51a2 2 0 0 1-1 1.74l-.15.09a2 2 0 0 0-.73 2.73l.22.38a2 2 0 0 0 2.73.73l.15-.08a2 2 0 0 1 2 0l.43.25a2 2 0 0 1 1 1.73V20a2 2 0 0 0 2 2h.44a2 2 0 0 0 2-2v-.18a2 2 0 0 1 1-1.73l.43-.25a2 2 0 0 1 2 0l.15.08a2 2 0 0 0 2.73-.73l.22-.39a2 2 0 0 0-.73-2.73l-.15-.08a2 2 0 0 1-1-1.74v-.5a2 2 0 0 1 1-1.74l.15-.09a2 2 0 0 0 .73-2.73l-.22-.38a2 2 0 0 0-2.73-.73l-.15.08a2 2 0 0 1-2 0l-.43-.25a2 2 0 0 1-1-1.73V4a2 2 0 0 0-2-2z'/%3E%3Ccircle cx='12' cy='12' r='3'/%3E%3C/svg%3E",+ "guide-operations":+ "data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='24' height='24' viewBox='0 0 24 24' fill='none' stroke='currentColor' stroke-width='1.5' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpolyline points='4 17 10 11 4 5'/%3E%3Cline x1='12' x2='20' y1='19' y2='19'/%3E%3C/svg%3E",);/* Generate icon styles for each sidebar item */diff --git a/docs/src/pages/index.tsx b/docs/src/pages/index.tsxindex 168173cd..ffee6f72 100644--- a/docs/src/pages/index.tsx+++ b/docs/src/pages/index.tsx@@ -1,56 +1,76 @@-import React, { ReactNode } from 'react';-import styles from './index.module.css';-import LayoutProvider from '@theme/Layout/Provider';-import Footer from '@theme/Footer';-import Link from '@docusaurus/Link';+import React, { ReactNode } from "react";+import styles from "./index.module.css";+import LayoutProvider from "@theme/Layout/Provider";+import Footer from "@theme/Footer";+import Link from "@docusaurus/Link";class HomeNav extends React.Component {- render() {- return <div className={styles.homeNav}>- <div><Link to={"/docs/getting-started"}>Docs</Link></div>- <div><Link to={"/guides/guide-custom-vulnerability"}>Guides</Link></div>- <div><a href="https://github.com/confident-ai/deepteam" target="_blank">Github</a></div>- <div><a href="https://confident-ai.com/blog" target="_blank">Blog</a></div>- {/* <div><Link to="/guides/guides-rag-evaluation">Guides</Link></div> */}- {/* <div className={styles.canHide}>+ render() {+ return (+ <div className={styles.homeNav}>+ <div>+ <Link to={"/docs/getting-started"}>Docs</Link>+ </div>+ <div>+ <Link to={"/guides/guide-agentic-ai-red-teaming"}>Guides</Link>+ </div>+ <div>+ <a href="https://github.com/confident-ai/deepteam" target="_blank">+ Github+ </a>+ </div>+ <div>+ <a href="https://confident-ai.com/blog" target="_blank">+ Blog+ </a>+ </div>+ {/* <div><Link to="/guides/guides-rag-evaluation">Guides</Link></div> */}+ {/* <div className={styles.canHide}><div><a href="https://github.com/confident-ai/deepteam" target="_blank">Github</a></div><div><a href="https://confident-ai.com/blog" target="_blank">Blog</a></div></div> */}- </div>- }+ </div>+ );+ }}class ConfidentEnvelope extends React.Component {handleConfident = () => {- window.open('https://confident-ai.com', '_blank');- }+ window.open("https://confident-ai.com", "_blank");+ };render() {- return <div className={styles.letterContainer} onClick={this.handleConfident}>- <div className={styles.letterImage}>- <div className={styles.animatedMail}>- <div className={styles.backFold}></div>- <div className={styles.letter}>- <div className={styles.letterBorder}></div>- <div className={styles.letterTitle}>Delivered by</div>- <div className={styles.letterContentContainer}>- <img src="icons/red-logo.svg" style={{width: "30px", height: "30px"}}/>- <div className={styles.letterContext}>- <span className="lexend-deca" style={{fontSize: "16px"}}>Confident AI</span>+ return (+ <div className={styles.letterContainer} onClick={this.handleConfident}>+ <div className={styles.letterImage}>+ <div className={styles.animatedMail}>+ <div className={styles.backFold}></div>+ <div className={styles.letter}>+ <div className={styles.letterBorder}></div>+ <div className={styles.letterTitle}>Delivered by</div>+ <div className={styles.letterContentContainer}>+ <img+ src="icons/red-logo.svg"+ style={{ width: "30px", height: "30px" }}+ />+ <div className={styles.letterContext}>+ <span className="lexend-deca" style={{ fontSize: "16px" }}>+ Confident AI+ </span>+ </div>+ </div>+ <div className={styles.letterStamp}>+ <div className={styles.letterStampInner}></div>+ </div></div>+ <div className={styles.topFold}></div>+ <div className={styles.body}></div>+ <div className={styles.leftFold}></div></div>- <div className={styles.letterStamp}>- <div className={styles.letterStampInner}></div>- </div>- </div>- <div className={styles.topFold}>+ <div className={styles.shadow}></div></div>- <div className={styles.body}></div>- <div className={styles.leftFold}></div></div>- <div className={styles.shadow}></div>- </div>- </div>+ );}}@@ -62,73 +82,80 @@ interface FeatureCardProps {class FeatureCard extends React.Component<FeatureCardProps> {render() {- const { title, link, description } = this.props;+ const { title, link, description } = this.props;- return (- <Link to={link} className={styles.featureCard}>- <div className={styles.featureCardContainer}>- <span className={styles.title}>{title}<img src="icons/right-arrow.svg" /></span>- </div>- <p className={styles.description}>{description}</p>- </Link>- );+ return (+ <Link to={link} className={styles.featureCard}>+ <div className={styles.featureCardContainer}>+ <span className={styles.title}>+ {title}+ <img src="icons/right-arrow.svg" />+ </span>+ </div>+ <p className={styles.description}>{description}</p>+ </Link>+ );}}class Index extends React.Component {handleConfident = () => {- window.open('https://confident-ai.com', '_blank');- }+ window.open("https://confident-ai.com", "_blank");+ };- render() {- return (- <div className={styles.mainMainContainer}>- <div className={styles.mainContainer}>- <div className={styles.mainLeftContainer}>- <img src="icons/DeepTeam.svg" />- <div className={styles.contentContainer}>- <h1>{`> the open-source LLM red teaming framework_`}</h1>- <div className={styles.ctaContainer}>- <Link to={"/docs/getting-started"} className={styles.button}>Get Started</Link>- {/* <a href={"https://confident-ai.com"} className={styles.confidentLink}>+ render() {+ return (+ <div className={styles.mainMainContainer}>+ <div className={styles.mainContainer}>+ <div className={styles.mainLeftContainer}>+ <img src="icons/DeepTeam.svg" />+ <div className={styles.contentContainer}>+ <h1>{`> the open-source LLM red teaming framework_`}</h1>+ <div className={styles.ctaContainer}>+ <Link to={"/docs/getting-started"} className={styles.button}>+ Get Started+ </Link>+ {/* <a href={"https://confident-ai.com"} className={styles.confidentLink}><span>Try the DeepTeam Platform</span><img className={styles.newTabIcon} src="icons/new-tab.svg"/></a> */}- </div></div></div>- <ConfidentEnvelope />- </div>- <div className={styles.featuresContainer}>- <FeatureCard- title="Detect 40+ LLM Vulnerabilities"- link="/docs/red-teaming-vulnerabilities"- description="Automatically scan for vulnerabilities such as bias, PII leakage, toxicity, etc."- />- <FeatureCard- title="SOTA Adersarial Attacks"- link="/docs/red-teaming-adversarial-attacks"- description="Prompt injections, gray box, etc. to jailbreak your LLM"- />- <FeatureCard- title="OWASP Top 10, NIST AI, etc."- link="/docs/red-teaming-owasp-top-10-for-llms"- description="OWASP Top 10 for LLMs, NIST AI, and so much more out of the box"- /></div>+ <ConfidentEnvelope />+ </div>+ <div className={styles.featuresContainer}>+ <FeatureCard+ title="Detect 40+ LLM Vulnerabilities"+ link="/docs/red-teaming-vulnerabilities"+ description="Automatically scan for vulnerabilities such as bias, PII leakage, toxicity, etc."+ />+ <FeatureCard+ title="SOTA Adersarial Attacks"+ link="/docs/red-teaming-adversarial-attacks"+ description="Prompt injections, gray box, etc. to jailbreak your LLM"+ />+ <FeatureCard+ title="OWASP Top 10, NIST AI, etc."+ link="/docs/red-teaming-owasp-top-10-for-llms"+ description="OWASP Top 10 for LLMs, NIST AI, and so much more out of the box"+ /></div>- );- }+ </div>+ );}+}export default function Home(props: any): ReactNode {- return <LayoutProvider>+ return (+ <LayoutProvider><div className={styles.mainRapper}><div className={styles.rapper}><HomeNav /><Index {...props} /></div></div>- <Footer/>- </LayoutProvider>;-};\ No newline at end of file+ <Footer />+ </LayoutProvider>+ );+}