Skip to content

DeepTeam is a framework to red team LLMs and AI agents.

.

penguine393e575b5a25
diff --git a/README.md b/README.md
index c6ab28bd..e69de29b 100644
--- a/README.md
+++ b/README.md
@@ -1,163 +0,0 @@
-<<<<<<< HEAD
-<p align="center">
- <img src="https://github.com/confident-ai/deepteam/blob/main/docs/static/img/deepteam.png" alt="DeepTeam Logo" width="100%">
-</p>
-
-<p align="center">
- <h1 align="center">The LLM Red Teaming Framework</h1>
-</p>
-
-<p align="center">
- <a href="https://discord.com/invite/a3K9c8GRGt">
- <img alt="discord-invite" src="https://dcbadge.vercel.app/api/server/a3K9c8GRGt?style=flat">
- </a>
-</p>
-
-<h4 align="center">
- <p>
- <a href="https://docs.confident-ai.com/docs/getting-started?utm_source=GitHub">Documentation</a> |
- <a href="#-metrics-and-features">Metrics and Features</a> |
- <a href="#-quickstart">Getting Started</a> |
- <a href="#-integrations">Integrations</a> |
- <a href="https://confident-ai.com?utm_source=GitHub">DeepEval Platform</a>
- <p>
-</h4>
-
-<p align="center">
- <a href="https://github.com/confident-ai/deepeval/releases">
- <img alt="GitHub release" src="https://img.shields.io/github/release/confident-ai/deepeval.svg?color=violet">
- </a>
- <a href="https://colab.research.google.com/drive/1PPxYEBa6eu__LquGoFFJZkhYgWVYE6kh?usp=sharing">
- <img alt="Try Quickstart in Colab" src="https://colab.research.google.com/assets/colab-badge.svg">
- </a>
- <a href="https://github.com/confident-ai/deepeval/blob/master/LICENSE.md">
- <img alt="License" src="https://img.shields.io/github/license/confident-ai/deepeval.svg?color=yellow">
- </a>
-</p>
-
-**DeepTeam** is a simple-to-use, open-source LLM red teaming framework, for security and safety testing large-language model systems. Deepteam incorporates the latest research to red team LLM systems such as RAG pipelines, agents, chatbots, and even just the LLM itself. DeepTeam uses adversarial attacks such as prompt injection, jailbreaking, ROT13 encoding, etc. to scan for vulnerabilities such as bias, PII leakage, and toxicity.
-
-Whether your LLM application is user-facing or not, DeepSeek or Claude, DeepTeam has you covered. With it, you can easily determine the security and and safet risks and vulnerability your LLM system possesses.
-
-
-> [!NOTE]
-> DeepTeam is powered by DeepEval.
-
-> Want to talk LLM safety and security vulnerabilities? [Come join our discord.](https://discord.com/invite/a3K9c8GRGt)
-
-<br />
-
-# ⚠️ Metrics, Risks, and Vulnerabilities 🚨
-
-- Large variety of ready-to-use LLM evaluation metrics (all with explanations) powered by **ANY** LLM of your choice, statistical methods, or NLP models that runs **locally on your machine**:
- - G-Eval
- - DAG ([deep acyclic graph](https://docs.confident-ai.com/docs/metrics-dag))
- - **RAG metrics:**
- - Answer Relevancy
- - Faithfulness
- - Contextual Recall
- - Contextual Precision
- - Contextual Relevancy
- - RAGAS
- - **Agentic metrics:**
-
-- [Red team your LLM application](https://docs.confident-ai.com/docs/red-teaming-introduction) for 40+ safety vulnerabilities in a few lines of code, including:
- - Toxicity
- - Bias
- - SQL Injection
- - etc., using advanced 10+ attack enhancement strategies such as prompt injections.
-
-
-<br />
-
-# 🚀 QuickStart
-
-Let's pretend your LLM application is a RAG based customer support chatbot; here's how DeepEval can help test what you've built.
-
-## Installation
-
-```
-pip install -U deepteam
-```
-
-## Writing your first test case
-
-Create a test file:
-
-```bash
-touch test_chatbot.py
-```
-
-Open `test_chatbot.py` and write your first test case using DeepEval:
-
-```python
-import pytest
-from deepeval import assert_test
-from deepeval.metrics import GEval
-from deepeval.test_case import LLMTestCase, LLMTestCaseParams
-
-def test_case():
- correctness_metric = GEval(
- name="Correctness",
- criteria="Determine if the 'actual output' is correct based on the 'expected output'.",
- evaluation_params=[LLMTestCaseParams.ACTUAL_OUTPUT, LLMTestCaseParams.EXPECTED_OUTPUT],
- threshold=0.5
- )
- test_case = LLMTestCase(
- input="What if these shoes don't fit?",
- # Replace this with the actual output from your LLM application
- actual_output="You have 30 days to get a full refund at no extra cost.",
- expected_output="We offer a 30-day full refund at no extra costs.",
- retrieval_context=["All customers are eligible for a 30 day full refund at no extra costs."]
- )
- assert_test(test_case, [correctness_metric])
-```
-Set your `OPENAI_API_KEY` as an environment variable (you can also evaluate using your own custom model, for more details visit [this part of our docs](https://docs.confident-ai.com/docs/metrics-introduction#using-a-custom-llm?utm_source=GitHub)):
-
-```
-export OPENAI_API_KEY="..."
-```
-
-And finally, run `test_chatbot.py` in the CLI:
-
-```
-deepeval test run test_chatbot.py
-```
-
-**Congratulations! Your test case should have passed ✅** Let's breakdown what happened.
-
-- The variable `input` mimics a user input, and `actual_output` is a placeholder for what your application's supposed to output based on this input.
-- The variable `expected_output` represents the ideal answer for a given `input`, and [`GEval`](https://docs.confident-ai.com/docs/metrics-llm-evals) is a research-backed metric provided by `deepeval` for you to evaluate your LLM output's on any custom custom with human-like accuracy.
-- In this example, the metric `criteria` is correctness of the `actual_output` based on the provided `expected_output`.
-- All metric scores range from 0 - 1, which the `threshold=0.5` threshold ultimately determines if your test have passed or not.
-
-[Read our documentation](https://docs.confident-ai.com/docs/getting-started?utm_source=GitHub) for more information on how to use additional metrics, create your own custom metrics, and tutorials on how to integrate with other tools like LangChain and LlamaIndex.
-
-<br />
-
-
-# Contributing
-
-Please read [CONTRIBUTING.md](https://github.com/confident-ai/deepteam/blob/main/CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.
-
-<br />
-
-# Roadmap
-
-Features:
-
-- [x] Red-Teaming
-
-<br />
-
-# Authors
-
-Built by the founders of Confident AI. Contact [email protected] for all enquiries.
-
-<br />
-
-# License
-
-DeepTeam is licensed under Apache 2.0 - see the [LICENSE.md](https://github.com/confident-ai/deepeval/blob/main/LICENSE.md) file for details.
-=======
->>>>>>> 27395ab (big changes)