Skip to content

Open-Sora: Democratizing Efficient Video Production for All

Python

update readme

Zangwei Zheng50b7c8691572
diff --git a/README.md b/README.md
index 50e594b..aaa5c99 100644
--- a/README.md
+++ b/README.md
@@ -46,7 +46,7 @@ Videos are downsampled to `.gif` for display. Click for original videos. Prompts
* πŸ“ Open-Sora-v1 released. Model weights are available [here](#model-weights). With only 400K video clips and 200 H800 days (compared with 152M samples in Stable Video Diffusion), we are able to generate 2s 512Γ—512 videos.
* βœ… Three stages training from an image diffusion model to a video diffusion model. We provide the weights for each stage.
* βœ… Support training acceleration including accelerated transformer, faster T5 and VAE, and sequence parallelism. Open-Sora improve **55%** training speed when training on 64x512x512 videos. Details locates at [acceleration.md](docs/acceleration.md).
-* βœ… We provide video cutting and captioning tools for data preprocessing. Instructions can be found [here](tools/data/README.md) and our data collection plan can be found at [datasets.md](docs/datasets.md).
+* βœ… We provide data preprocessing pipeline, including [downloading](/tools/datasets/README.md), [video cutting](/tools/scenedetect/README.md), and [captioning](/tools/caption/README.md) tools. Our data collection plan can be found at [datasets.md](docs/datasets.md).
* βœ… We find VQ-VAE from [VideoGPT](https://wilson1yan.github.io/videogpt/index.html) has a low quality and thus adopt a better VAE from [Stability-AI](https://huggingface.co/stabilityai/sd-vae-ft-mse-original). We also find patching in the time dimension deteriorates the quality. See our **[report](docs/report_v1.md)** for more discussions.
* βœ… We investigate different architectures including DiT, Latte, and our proposed STDiT. Our **STDiT** achieves a better trade-off between quality and speed. See our **[report](docs/report_v1.md)** for more discussions.
* βœ… Support clip and T5 text conditioning.