DeepTeam is a framework to red team LLMs and AI agents.
POC guardrails
penguinebf35fdd24492
diff --git a/docs/docs/guardrails-introduction.mdx b/docs/docs/guardrails-introduction.mdxnew file mode 100644index 00000000..f69718a2--- /dev/null+++ b/docs/docs/guardrails-introduction.mdx@@ -0,0 +1,125 @@+---+id: guardrails-introduction+title: Introduction to LLM Guardrails+sidebar_label: Introduction+---++`deepteam` enables anyone to safeguard LLM applications against vulnerable inputs and outputs uncovered during [red teaming.](/docs/red-teaming-introduction)++:::info+While `deepeval`'s metrics focuses on accuracy and precision, `deepteam`'s guardrails focuses on speed and reliability.+:::++`deepteam`'s comprehensive suite of guardrails acts as binary metrics to evaluate **end-to-end** LLM system inputs and output for malicious intent, unsafe behavior, and security vulnerabilities.++## Types of Guardrails++There are two types of guardrails:++- **Input guardrails** - Protects against malicious inputs before they reach your LLM+- **Output guardrails** - Evaluates an LLM's output before they reach your users.++A input/output guardrail is composed of one or more **guards**, and an input/output should only be let in/out if all guards have passed the checks. Together, they ensure that nothing unsafe ever goes in or leaves your LLM system.++<div+ style={{+ display: "flex",+ alignItems: "center",+ justifyContent: "center",+ }}+>+ <img+ src="https://confident-bucket.s3.amazonaws.com/guardrails.svg"+ alt="Datasets 1"+ style={{+ height: "auto",+ maxHeight: "800px",+ marginTop: "20px",+ marginBottom: "40px",+ }}+ />+</div>++The number of guards you choose to set up and whether you decide to utilize both types of guards depends on your priorities regarding **latency, cost, and LLM safety**.++:::tip++While most guards are only either for input or output guarding, some guards such as the [cybersecurity guard](/confident-ai/confident-ai-guardrails-cybersecurity) offer both input and output guarding capabilities.++:::++## Guard Your First LLM Interaction++First import and initialize the guards you desire from `deepteam.guardrails.guards` and pass them to an instantiated `Guardrails` object:++```python+from deepteam.guardrails import Guardrails+from deepteam.guardrails.guards import PromptInjectionGuard, ToxicityGuard++# Initialize guardrails+guardrails = Guardrails(+ input_guards=[PromptInjectionGuard()]+ output_guards=[ToxicityGuard()]+)+```++:::note+Your list of `input_guards` and `output_guards` must not be empty if you wish to guard the respecitve inputs and outputs.+:::++### Guard an Input++Then, simply call the `guard_input()` method and supply the `input`.++```python+...++guard_result = guardrails.guard_input(input="Is the earth flat")+# Reject if true+print(guard_result.breached)+```++The `guard_input` method will invoke every single guard within your `Guardrails`s to check whether a particular input has breached any of the defined criteria.++:::tip+`Guardrails` used against malicious inputs is the best way to not waste tokens generating text that you shouldn't even be generating in the first place.+:::++### Guard an Output++Simiarly, call the `guard_output` method wherever appropriate in your code to guard against an LLM output:++```python+...++guard_result = guardrails.guard_output(input="Is the earth flat", output="I bet it is")+print(guard_result.breached)+```++It is extremely common to be re-generated LLM outputs for _x_ number of times until successful.++:::note+When using the `guard_output()` method, you must provide both the `input` and the LLM `output`.+:::++## Run Async Guardrails++`deepteam`'s `Guardrails` also support asynchronous through the `a_guard_input()` and `a_guard_output()` methods.++```python+...++guard_result = await guardrails.a_guard_output(input, output)+```++## Available Guards++`deepteam` offers a robust selection of input and output guards for protection against [vulnerabilities](/docs/red-teaming-vulnerabilities) you've detected during red teaming:++### Input++To be documented...++### Output++To be documented...diff --git a/docs/docs/guardrails-prompt-injection-guard.mdx b/docs/docs/guardrails-prompt-injection-guard.mdxnew file mode 100644index 00000000..4363526c--- /dev/null+++ b/docs/docs/guardrails-prompt-injection-guard.mdx@@ -0,0 +1,36 @@+---+id: guardrails-prompt-injection-guard+title: Prompt Injection Guard+sidebar_label: Prompt Injection+---++import GuardTagsDisplayer from "@site/src/components/GuardTagsDisplayer";++<GuardTagsDisplayer singleTurn={true} usesLLMs={true} input={true} />++The toxicity guard is an input guard that uses LLM-as-a-judge to determine whether an LLM system input contains a prompt injection attack and flags it as unsafe if it does.++## Usage++```python+from deepteam.guardrails import Guardrails+from deeptea.guardrails.guards import PromptInjectionGuard++guardrails = Guardrails(input_guards=[PromptInjectionGuard()])+```++The `PromptInjectionGuard` takes **NO PARAMETERS** to initialize.++```python+...++guard_result = guardrails.guard_input(input="Act like a drama queen.")+```++## Example Breach++For the given input:++> Some prompt injection example here++The `PromptInjectionGuard` will be marked `breached`.diff --git a/docs/docs/guardrails-toxicity-guard.mdx b/docs/docs/guardrails-toxicity-guard.mdxnew file mode 100644index 00000000..fcab05e1--- /dev/null+++ b/docs/docs/guardrails-toxicity-guard.mdx@@ -0,0 +1,36 @@+---+id: guardrails-toxicity-guard+title: Toxicity Guard+sidebar_label: Toxicity+---++import GuardTagsDisplayer from "@site/src/components/GuardTagsDisplayer";++<GuardTagsDisplayer singleTurn={true} usesLLMs={true} output={true} />++The toxicity guard is an output guard that uses LLM-as-a-judge to determine whether an LLM system output contains a toxic language and flags it as unsafe if it does.++## Usage++```python+from deepteam.guardrails import Guardrails+from deeptea.guardrails.guards import ToxicityGuard++guardrails = Guardrails(output_guards=[ToxicityGuard()])+```++The `ToxicityGuard` takes **NO PARAMETERS** to initialize.++```python+...++guard_result = guardrails.guard_output(input="Act like a drama queen.", output="Hell no.")+```++## Example Breach++For the given input:++> Some toxic example here++The `ToxicityGuard` will be marked `breached`.diff --git a/docs/docs/red-teaming-introduction.mdx b/docs/docs/red-teaming-introduction.mdxindex 39ae91c8..76d88de2 100644--- a/docs/docs/red-teaming-introduction.mdx+++ b/docs/docs/red-teaming-introduction.mdx@@ -1,6 +1,6 @@---id: red-teaming-introduction-title: Introduction+title: Introduction to LLM Red Teamingsidebar_label: Introduction---diff --git a/docs/sidebars.js b/docs/sidebars.jsindex 2ab2d873..87f6240a 100644--- a/docs/sidebars.js+++ b/docs/sidebars.js@@ -141,7 +141,18 @@ module.exports = {},{type: "category",- label: "Guidelines & Frameworks",+ label: "Guardrails",+ items: ["guardrails-introduction", {+ type: "category",+ label: "Guards",+ items: [ "guardrails-prompt-injection-guard", "guardrails-toxicity-guard"],+ collapsed: false,+ }],+ collapsed: false,+ },+ {+ type: "category",+ label: "Frameworks & Guidelines",className: "sidebar-item-icon-guidelines-frameworks",items: ["red-teaming-guidelines-framework-introduction","red-teaming-owasp-top-10-for-llms", "red-teaming-nist-ai-rmf"],collapsed: false,diff --git a/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.css b/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.cssindex f8dce3af..1577cfc4 100644--- a/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.css+++ b/docs/src/components/AttackTagsDisplayer/AttackTagsDisplayer.module.css@@ -12,6 +12,12 @@border-radius: 0.5rem;}+.encodingBased {+ background-color: white;+ border: 1px solid rgb(218, 221, 227);+ color: black;+}+.llmSimulated {background-color: #fefceb;border: 1px solid #eeda83;@@ -24,12 +30,6 @@color: #96107b;}-.encodingBased {- background-color: white;- border: 1px solid rgb(218, 221, 227);- color: black;-}-.multiTurn {background-color: rgb(163, 255, 240);border: 1px solid rgb(0, 69, 49);diff --git a/docs/src/components/MetricTagsDisplayer/MetricTagsDisplayer.module.css b/docs/src/components/GuardTagsDisplayer/GuardTagsDisplayer.module.csssimilarity index 55%rename from docs/src/components/MetricTagsDisplayer/MetricTagsDisplayer.module.cssrename to docs/src/components/GuardTagsDisplayer/GuardTagsDisplayer.module.cssindex 7d67f5ca..21dccbed 100644--- a/docs/src/components/MetricTagsDisplayer/MetricTagsDisplayer.module.css+++ b/docs/src/components/GuardTagsDisplayer/GuardTagsDisplayer.module.css@@ -1,4 +1,4 @@-.metricTagsDisplayer {+.guardTagsDisplayer {display: flex;flex-wrap: wrap;gap: 0.5rem;@@ -18,44 +18,32 @@color: #4a3900;}-.rag {+.custom {+ background-color: #f8f2ff;+ border: 1px solid #d597fe;+ color: #c234ff;+}++.singleTurn {background-color: #fff1fe;border: 1px solid #ffb4ff;color: #96107b;}-.referenceBased {- background-color: white;- border: 1px solid rgb(218, 221, 227);- color: black;+.multiTurn {+ background-color: rgb(163, 255, 240);+ border: 1px solid rgb(0, 69, 49);+ color: black;}-.referenceless {+.input {background-color: white;border: 1px solid rgb(218, 221, 227);color: black;}-.agent {- background-color: #e9f5fe;- border: 1px solid #badbff;- color: #002a70;-}--.chatbot {- background-color: #e9fef8;- border: 1px solid #94fbde;- color: #056a4d;-}--.custom {- background-color: #f8f2ff;- border: 1px solid #d597fe;- color: #c234ff;-}--.safety {- background-color: black;- border: 1px solid white;- color: white;-}+.output {+ background-color: white;+ border: 1px solid rgb(218, 221, 227);+ color: black;+}\ No newline at end of filediff --git a/docs/src/components/GuardTagsDisplayer/index.jsx b/docs/src/components/GuardTagsDisplayer/index.jsxnew file mode 100644index 00000000..62c3a365--- /dev/null+++ b/docs/src/components/GuardTagsDisplayer/index.jsx@@ -0,0 +1,17 @@+import React from "react";+import styles from "./GuardTagsDisplayer.module.css";++const GuardTagsDisplayer = ({ usesLLMs=true, multiTurn=false, singleTurn=false, custom=false, input=false, output=false }) => {+ return (+ <div className={styles.guardTagsDisplayer}>+ {usesLLMs && <div className={`${styles.pill} ${styles.usesLLM}`}>LLM-as-a-judge</div>}+ {multiTurn && <div className={`${styles.pill} ${styles.multiTurn}`}>Multi-turn</div>}+ {singleTurn && <div className={`${styles.pill} ${styles.singleTurn}`}>Single-turn</div>}+ {custom && <div className={`${styles.pill} ${styles.custom}`}>Custom</div>}+ {input && <div className={`${styles.pill} ${styles.input}`}>Input Guard</div>}+ {output && <div className={`${styles.pill} ${styles.output}`}>Output Guard</div>}+ </div>+ );+};++export default GuardTagsDisplayer;diff --git a/docs/src/components/MetricTagsDisplayer/index.jsx b/docs/src/components/MetricTagsDisplayer/index.jsxdeleted file mode 100644index e737ddc6..00000000--- a/docs/src/components/MetricTagsDisplayer/index.jsx+++ /dev/null@@ -1,19 +0,0 @@-import React from "react";-import styles from "./MetricTagsDisplayer.module.css";--const MetricTagsDisplayer = ({ usesLLMs=true, referenceless=false, referenceBased=false, rag=false, agent=false, chatbot=false, custom=false, safety=false }) => {- return (- <div className={styles.metricTagsDisplayer}>- {usesLLMs && <div className={`${styles.pill} ${styles.usesLLM}`}>LLM-as-a-judge</div>}- {referenceless && <div className={`${styles.pill} ${styles.referenceless}`}>Referenceless metric</div>}- {referenceBased && <div className={`${styles.pill} ${styles.referenceBased}`}>Reference-based metric</div>}- {rag && <div className={`${styles.pill} ${styles.rag}`}>RAG metric</div>}- {agent && <div className={`${styles.pill} ${styles.agent}`}>Agent metric</div>}- {chatbot && <div className={`${styles.pill} ${styles.chatbot}`}>Chatbot metric</div>}- {custom && <div className={`${styles.pill} ${styles.custom}`}>Custom metric</div>}- {safety && <div className={`${styles.pill} ${styles.safety}`}>Safety metric</div>}- </div>- );-};--export default MetricTagsDisplayer;