Skip to content

Open-Sora: Democratizing Efficient Video Production for All

Python

update datasets.md

Tom Young5218d5a928f6
diff --git a/docs/datasets.md b/docs/datasets.md
index ca09635..e38e6a4 100644
--- a/docs/datasets.md
+++ b/docs/datasets.md
@@ -1,16 +1,28 @@
# Datasets
-For Open-Sora 1.1, we conduct mixed training with both images and videos. The main datasets we use are listed below.
+For Open-Sora 1.2, we conduct mixed training with both images and videos. The main datasets we use are listed below.
Please refer to [README](/README.md#data-processing) for data processing.
## Video
+### Webvid-10M
+
+[Webvid-10M](https://github.com/m-bain/webvid) contains 10 million video-text pairs scraped from the stock footage sites.
+We first train the model on this dataset (40k hours) for 30k steps (2 epochs).
+
### Panda-70M
[Panda-70M](https://github.com/snap-research/Panda-70M) is a large-scale dataset with 70M video-caption pairs.
We use the [training-10M subset](https://github.com/snap-research/Panda-70M/tree/main/dataset_dataloading) for training,
which contains ~10M videos of better quality.
+### Mixkit
+[Mixkit](https://mixkit.co/) is a video website where we obtained 9k videos.
+
+### Pixabay
+[Pixabay](https://pixabay.com/videos/) is video website where we obtained 60.5k videos.
+
+
### Pexels
[Pexels](https://www.pexels.com/) is a popular online platform that provides high-quality stock photos, videos, and music for free.
@@ -28,6 +40,16 @@ The dataset is proposed for super-resolution tasks. We use the dataset for HQ fi
The caption is generated by BLIP-2.
We find the scene and the text quality are relatively poor. For OpenSora 1.0, we only use ~350K samples from this dataset.
+### MiraData
+
+[MiraData](https://github.com/mira-space/MiraData): a high-quality dataset with 77k long videos, mainly from games and city/scenic exploration.
+
+
+### Vript
+
+[Vript](https://github.com/mutonix/Vript/tree/main): a densely annotated dataset of 400k videos.
+
+
## Image
### Midjourney-v5-1.7M