Open-Sora: Democratizing Efficient Video Production for All
Python
update datasets.md
Tom Young5218d5a928f6
diff --git a/docs/datasets.md b/docs/datasets.mdindex ca09635..e38e6a4 100644--- a/docs/datasets.md+++ b/docs/datasets.md@@ -1,16 +1,28 @@# Datasets-For Open-Sora 1.1, we conduct mixed training with both images and videos. The main datasets we use are listed below.+For Open-Sora 1.2, we conduct mixed training with both images and videos. The main datasets we use are listed below.Please refer to [README](/README.md#data-processing) for data processing.## Video+### Webvid-10M++[Webvid-10M](https://github.com/m-bain/webvid) contains 10 million video-text pairs scraped from the stock footage sites.+We first train the model on this dataset (40k hours) for 30k steps (2 epochs).+### Panda-70M[Panda-70M](https://github.com/snap-research/Panda-70M) is a large-scale dataset with 70M video-caption pairs.We use the [training-10M subset](https://github.com/snap-research/Panda-70M/tree/main/dataset_dataloading) for training,which contains ~10M videos of better quality.+### Mixkit+[Mixkit](https://mixkit.co/) is a video website where we obtained 9k videos.++### Pixabay+[Pixabay](https://pixabay.com/videos/) is video website where we obtained 60.5k videos.++### Pexels[Pexels](https://www.pexels.com/) is a popular online platform that provides high-quality stock photos, videos, and music for free.@@ -28,6 +40,16 @@ The dataset is proposed for super-resolution tasks. We use the dataset for HQ fiThe caption is generated by BLIP-2.We find the scene and the text quality are relatively poor. For OpenSora 1.0, we only use ~350K samples from this dataset.+### MiraData++[MiraData](https://github.com/mira-space/MiraData): a high-quality dataset with 77k long videos, mainly from games and city/scenic exploration.+++### Vript++[Vript](https://github.com/mutonix/Vript/tree/main): a densely annotated dataset of 400k videos.++## Image### Midjourney-v5-1.7M