Skip to content

DeepTeam is a framework to red team LLMs and AI agents.

POC guardrails

penguinebf35fdd24492
diff --git a/docs/docs/guardrails-introduction.mdx b/docs/docs/guardrails-introduction.mdx
new file mode 100644
index 00000000..f69718a2
--- /dev/null
+++ b/docs/docs/guardrails-introduction.mdx
@@ -0,0 +1,125 @@
+---
+id: guardrails-introduction
+title: Introduction to LLM Guardrails
+sidebar_label: Introduction
+---
+
+`deepteam` enables anyone to safeguard LLM applications against vulnerable inputs and outputs uncovered during [red teaming.](/docs/red-teaming-introduction)
+
+:::info
+While `deepeval`'s metrics focuses on accuracy and precision, `deepteam`'s guardrails focuses on speed and reliability.
+:::
+
+`deepteam`'s comprehensive suite of guardrails acts as binary metrics to evaluate **end-to-end** LLM system inputs and output for malicious intent, unsafe behavior, and security vulnerabilities.
+
+## Types of Guardrails
+
+There are two types of guardrails:
+
+- **Input guardrails** - Protects against malicious inputs before they reach your LLM
+- **Output guardrails** - Evaluates an LLM's output before they reach your users.
+
+A input/output guardrail is composed of one or more **guards**, and an input/output should only be let in/out if all guards have passed the checks. Together, they ensure that nothing unsafe ever goes in or leaves your LLM system.
+
+<div
+ style={{
+ display: "flex",
+ alignItems: "center",
+ justifyContent: "center",
+ }}
+>
+ <img
+ src="https://confident-bucket.s3.amazonaws.com/guardrails.svg"
+ alt="Datasets 1"
+ style={{
+ height: "auto",
+ maxHeight: "800px",
+ marginTop: "20px",
+ marginBottom: "40px",
+ }}
+ />
+</div>
+
+The number of guards you choose to set up and whether you decide to utilize both types of guards depends on your priorities regarding **latency, cost, and LLM safety**.
+
+:::tip
+
+While most guards are only either for input or output guarding, some guards such as the [cybersecurity guard](/confident-ai/confident-ai-guardrails-cybersecurity) offer both input and output guarding capabilities.
+
+:::
+
+## Guard Your First LLM Interaction
+
+First import and initialize the guards you desire from `deepteam.guardrails.guards` and pass them to an instantiated `Guardrails` object:
+
+```python
+from deepteam.guardrails import Guardrails
+from deepteam.guardrails.guards import PromptInjectionGuard, ToxicityGuard
+
+# Initialize guardrails
+guardrails = Guardrails(
+ input_guards=[PromptInjectionGuard()]
+ output_guards=[ToxicityGuard()]
+)
+```
+
+:::note
+Your list of `input_guards` and `output_guards` must not be empty if you wish to guard the respecitve inputs and outputs.
+:::
+
+### Guard an Input
+
+Then, simply call the `guard_input()` method and supply the `input`.
+
+```python
+...
+
+guard_result = guardrails.guard_input(input="Is the earth flat")
+# Reject if true
+print(guard_result.breached)
+```
+
+The `guard_input` method will invoke every single guard within your `Guardrails`s to check whether a particular input has breached any of the defined criteria.
+
+:::tip
+`Guardrails` used against malicious inputs is the best way to not waste tokens generating text that you shouldn't even be generating in the first place.
+:::
+
+### Guard an Output
+
+Simiarly, call the `guard_output` method wherever appropriate in your code to guard against an LLM output:
+
+```python
+...
+
+guard_result = guardrails.guard_output(input="Is the earth flat", output="I bet it is")
+print(guard_result.breached)
+```
+
+It is extremely common to be re-generated LLM outputs for _x_ number of times until successful.
+
+:::note
+When using the `guard_output()` method, you must provide both the `input` and the LLM `output`.
+:::
+
+## Run Async Guardrails
+
+`deepteam`'s `Guardrails` also support asynchronous through the `a_guard_input()` and `a_guard_output()` methods.
+
+```python
+...
+
+guard_result = await guardrails.a_guard_output(input, output)
+```
+
+## Available Guards
+
+`deepteam` offers a robust selection of input and output guards for protection against [vulnerabilities](/docs/red-teaming-vulnerabilities) you've detected during red teaming:
+
+### Input
+
+To be documented...
+
+### Output
+
+To be documented...
diff --git a/docs/docs/guardrails-prompt-injection-guard.mdx b/docs/docs/guardrails-prompt-injection-guard.mdx
new file mode 100644
index 00000000..4363526c
--- /dev/null
+++ b/docs/docs/guardrails-prompt-injection-guard.mdx
@@ -0,0 +1,36 @@
+---
+id: guardrails-prompt-injection-guard
+title: Prompt Injection Guard
+sidebar_label: Prompt Injection
+---
+
+import GuardTagsDisplayer from "@site/src/components/GuardTagsDisplayer";
+
+<GuardTagsDisplayer singleTurn={true} usesLLMs={true} input={true} />
+
+The toxicity guard is an input guard that uses LLM-as-a-judge to determine whether an LLM system input contains a prompt injection attack and flags it as unsafe if it does.
+
+## Usage
+
+```python
+from deepteam.guardrails import Guardrails
+from deeptea.guardrails.guards import PromptInjectionGuard
+
+guardrails = Guardrails(input_guards=[PromptInjectionGuard()])
+```
+
+The `PromptInjectionGuard` takes **NO PARAMETERS** to initialize.
+
+```python
+...
+
+guard_result = guardrails.guard_input(input="Act like a drama queen.")
+```
+
+## Example Breach
+
+For the given input:
+
+> Some prompt injection example here
+
+The `PromptInjectionGuard` will be marked `breached`.
diff --git a/docs/docs/guardrails-toxicity-guard.mdx b/docs/docs/guardrails-toxicity-guard.mdx
new file mode 100644
index 00000000..fcab05e1
--- /dev/null
+++ b/docs/docs/guardrails-toxicity-guard.mdx
@@ -0,0 +1,36 @@
+---
+id: guardrails-toxicity-guard
+title: Toxicity Guard
+sidebar_label: Toxicity
+---
+
+import GuardTagsDisplayer from "@site/src/components/GuardTagsDisplayer";
+
+<GuardTagsDisplayer singleTurn={true} usesLLMs={true} output={true} />
+
+The toxicity guard is an output guard that uses LLM-as-a-judge to determine whether an LLM system output contains a toxic language and flags it as unsafe if it does.
+
+## Usage
+
+```python
+from deepteam.guardrails import Guardrails
+from deeptea.guardrails.guards import ToxicityGuard
+
+guardrails = Guardrails(output_guards=[ToxicityGuard()])
+```
+
+The `ToxicityGuard` takes **NO PARAMETERS** to initialize.
+
+```python
+...
+
+guard_result = guardrails.guard_output(input="Act like a drama queen.", output="Hell no.")
+```
+
+## Example Breach
+
+For the given input:
+
+> Some toxic example here
+
+The `ToxicityGuard` will be marked `breached`.
diff --git a/docs/docs/red-teaming-introduction.mdx b/docs/docs/red-teaming-introduction.mdx
index 39ae91c8..76d88de2 100644
--- a/docs/docs/red-teaming-introduction.mdx
+++ b/docs/docs/red-teaming-introduction.mdx
@@ -1,6 +1,6 @@
---
id: red-teaming-introduction
-title: Introduction
+title: Introduction to LLM Red Teaming
sidebar_label: Introduction
---
diff --git a/docs/sidebars.js b/docs/sidebars.js
index 2ab2d873..87f6240a 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -141,7 +141,18 @@ module.exports = {
},
{
type: "category",
- label: "Guidelines & Frameworks",
+ label: "Guardrails",
+ items: ["guardrails-introduction", {
+ type: "category",
+ label: "Guards",
+ items: [ "guardrails-prompt-injection-guard", "guardrails-toxicity-guard"],
+ collapsed: false,
+ }],
+ collapsed: false,
+ },
+ {
+ type: "category",
+ label: "Frameworks & Guidelines",
className: "sidebar-item-icon-guidelines-frameworks",
items: ["red-teaming-guidelines-framework-introduction","red-teaming-owasp-top-10-for-llms", "red-teaming-nist-ai-rmf"],
collapsed: false,
diff --git a/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.css b/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.css
index f8dce3af..1577cfc4 100644
--- a/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.css
+++ b/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.css
@@ -12,6 +12,12 @@
border-radius: 0.5rem;
}
+.encodingBased {
+ background-color: white;
+ border: 1px solid rgb(218, 221, 227);
+ color: black;
+}
+
.llmSimulated {
background-color: #fefceb;
border: 1px solid #eeda83;
@@ -24,12 +30,6 @@
color: #96107b;
}
-.encodingBased {
- background-color: white;
- border: 1px solid rgb(218, 221, 227);
- color: black;
-}
-
.multiTurn {
background-color: rgb(163, 255, 240);
border: 1px solid rgb(0, 69, 49);
diff --git a/docs/src/components/MetricTagsDisplayer/MetricTagsDisplayer.module.css b/docs/src/components/GuardTagsDisplayer/GuardTagsDisplayer.module.css
similarity index 55%
rename from docs/src/components/MetricTagsDisplayer/MetricTagsDisplayer.module.css
rename to docs/src/components/GuardTagsDisplayer/GuardTagsDisplayer.module.css
index 7d67f5ca..21dccbed 100644
--- a/docs/src/components/MetricTagsDisplayer/MetricTagsDisplayer.module.css
+++ b/docs/src/components/GuardTagsDisplayer/GuardTagsDisplayer.module.css
@@ -1,4 +1,4 @@
-.metricTagsDisplayer {
+.guardTagsDisplayer {
display: flex;
flex-wrap: wrap;
gap: 0.5rem;
@@ -18,44 +18,32 @@
color: #4a3900;
}
-.rag {
+.custom {
+ background-color: #f8f2ff;
+ border: 1px solid #d597fe;
+ color: #c234ff;
+}
+
+.singleTurn {
background-color: #fff1fe;
border: 1px solid #ffb4ff;
color: #96107b;
}
-.referenceBased {
- background-color: white;
- border: 1px solid rgb(218, 221, 227);
- color: black;
+.multiTurn {
+ background-color: rgb(163, 255, 240);
+ border: 1px solid rgb(0, 69, 49);
+ color: black;
}
-.referenceless {
+.input {
background-color: white;
border: 1px solid rgb(218, 221, 227);
color: black;
}
-.agent {
- background-color: #e9f5fe;
- border: 1px solid #badbff;
- color: #002a70;
-}
-
-.chatbot {
- background-color: #e9fef8;
- border: 1px solid #94fbde;
- color: #056a4d;
-}
-
-.custom {
- background-color: #f8f2ff;
- border: 1px solid #d597fe;
- color: #c234ff;
-}
-
-.safety {
- background-color: black;
- border: 1px solid white;
- color: white;
-}
+.output {
+ background-color: white;
+ border: 1px solid rgb(218, 221, 227);
+ color: black;
+}
\ No newline at end of file
diff --git a/docs/src/components/GuardTagsDisplayer/index.jsx b/docs/src/components/GuardTagsDisplayer/index.jsx
new file mode 100644
index 00000000..62c3a365
--- /dev/null
+++ b/docs/src/components/GuardTagsDisplayer/index.jsx
@@ -0,0 +1,17 @@
+import React from "react";
+import styles from "./GuardTagsDisplayer.module.css";
+
+const GuardTagsDisplayer = ({ usesLLMs=true, multiTurn=false, singleTurn=false, custom=false, input=false, output=false }) => {
+ return (
+ <div className={styles.guardTagsDisplayer}>
+ {usesLLMs && <div className={`${styles.pill} ${styles.usesLLM}`}>LLM-as-a-judge</div>}
+ {multiTurn && <div className={`${styles.pill} ${styles.multiTurn}`}>Multi-turn</div>}
+ {singleTurn && <div className={`${styles.pill} ${styles.singleTurn}`}>Single-turn</div>}
+ {custom && <div className={`${styles.pill} ${styles.custom}`}>Custom</div>}
+ {input && <div className={`${styles.pill} ${styles.input}`}>Input Guard</div>}
+ {output && <div className={`${styles.pill} ${styles.output}`}>Output Guard</div>}
+ </div>
+ );
+};
+
+export default GuardTagsDisplayer;
diff --git a/docs/src/components/MetricTagsDisplayer/index.jsx b/docs/src/components/MetricTagsDisplayer/index.jsx
deleted file mode 100644
index e737ddc6..00000000
--- a/docs/src/components/MetricTagsDisplayer/index.jsx
+++ /dev/null
@@ -1,19 +0,0 @@
-import React from "react";
-import styles from "./MetricTagsDisplayer.module.css";
-
-const MetricTagsDisplayer = ({ usesLLMs=true, referenceless=false, referenceBased=false, rag=false, agent=false, chatbot=false, custom=false, safety=false }) => {
- return (
- <div className={styles.metricTagsDisplayer}>
- {usesLLMs && <div className={`${styles.pill} ${styles.usesLLM}`}>LLM-as-a-judge</div>}
- {referenceless && <div className={`${styles.pill} ${styles.referenceless}`}>Referenceless metric</div>}
- {referenceBased && <div className={`${styles.pill} ${styles.referenceBased}`}>Reference-based metric</div>}
- {rag && <div className={`${styles.pill} ${styles.rag}`}>RAG metric</div>}
- {agent && <div className={`${styles.pill} ${styles.agent}`}>Agent metric</div>}
- {chatbot && <div className={`${styles.pill} ${styles.chatbot}`}>Chatbot metric</div>}
- {custom && <div className={`${styles.pill} ${styles.custom}`}>Custom metric</div>}
- {safety && <div className={`${styles.pill} ${styles.safety}`}>Safety metric</div>}
- </div>
- );
-};
-
-export default MetricTagsDisplayer;