Open-Sora: Democratizing Efficient Video Production for All
Python
Dev/pxy (#67)
xyupengc6cbc7a6bfc3
diff --git a/docs/data_processing.md b/docs/data_processing.mdindex 47532e8..21c55e3 100644--- a/docs/data_processing.md+++ b/docs/data_processing.md@@ -1,9 +1,20 @@# Data Processing+We establish a complete pipeline for video/image data processing. The pipeline is shown below.-## Dataset Management+-## Scene Detection and Video Splitting+First, raw videos,+either from the Internet or public datasets, are split into shorter clips based on scene detection.+Then, we evaluate these videos by predicting multiple scores using existing models. We first predict the aesthetic score+and the optical flow score for a video. We also conduct OCR to detect texts in the video. Only videos with satisfactory+evaluation results are sent to the next step for captioning. After captioning, the matching score is also calculated as+an assessment of video-text alignment. Finally, we filter samples based on the matching score and+conduct camera motion detection for the remaining samples.+In summary, our pipeline produces video-text pairs which have high aesthetic quality, large video motion and strong+semantic consistency.-## Scoring and Filtering--## Captioning\ No newline at end of file+For more information, please refer to:+- [Dataset Management](https://github.com/hpcaitech/Open-Sora-dev/blob/dev/v1.1/tools/datasets/README.md)+- [Scene Detection and Video Splitting](https://github.com/hpcaitech/Open-Sora-dev/blob/dev/v1.1/tools/scene_cut/README.md)+- [Scoring and Filtering](https://github.com/hpcaitech/Open-Sora-dev/blob/dev/v1.1/tools/scoring/README.md)+- [Captioning](https://github.com/hpcaitech/Open-Sora-dev/blob/dev/v1.1/tools/caption/README.md)\ No newline at end of file