Skip to content

Open-Sora: Democratizing Efficient Video Production for All

Python

[doc] add summary and benchmark (#24)

binmakeswell901f43726cbf
diff --git a/README.md b/README.md
index b17d404..8058a0b 100644
--- a/README.md
+++ b/README.md
@@ -14,8 +14,21 @@
- [πŸͺ„ Acknowledgement](#-acknowledgement)
## πŸ“ Overview
+Open-Sora is an open-source project that provides a high-performance implementation of the development pipeline that Sora might use powered by [Colossal-AI](https://github.com/hpcaitech/ColossalAI), including:
-This repository is an unofficial implementation of OpenAI's Sora. We built this based on the [facebookresearch/DiT](https://github.com/facebookresearch/DiT) repository.
+- Provide **a complete Sora reproduction architecture solution**, including the whole process from data processing to training and inference.
+- Supports **dynamic resolution**, training can directly train any resolution of the video, without scaling.
+- Supports **multiple model structures**. Since the actual model structure of Sora is unknown, we realize three common multimodal model structures such as adaLN-zero, cross attention, and in-context conditioning (token concat).
+- Supports **multiple video compression methods**. Users can choose to use original video, VQVAE (video native model), SD-VAE (image native model) for training.
+- Supports **multiple parallel training optimizations**. Including the AI large model system optimization capability combined with Colossal-AI, and hybrid sequence parallelism with Ulysses and FastSeq.
+
+<p id="diffusion_demo" align="center">
+<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/applications/sora/open-sora-1.png" width=800/>
+</p>
+
+<p id="diffusion_demo" align="center">
+<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/applications/sora/open-sora-2.png" width=800/>
+</p>
## πŸ“‚ Dataset Preparation