Open-Sora: Democratizing Efficient Video Production for All
Python
Dev/pxy (#37)
xyupeng7da8be989a42
diff --git a/tools/datasets/README.md b/tools/datasets/README.mdindex c24a0be..284a8fb 100644--- a/tools/datasets/README.md+++ b/tools/datasets/README.md@@ -17,10 +17,24 @@ After preparing the raw dataset according to the [instructions](/docs/datasets.m## Dataset Format-All dataset should be provided in a CSV file, which is used both for training and data preprocessing.. The CSV file should only contain the following columns (can be optional).+All dataset should be provided in a CSV file, which is used both for training and data preprocessing. The CSV file should only contain the following columns (some are optional).+- `path`: the relative/absolute path or url to the image or video file. The only required column.+- `text`: the caption or description of the image or video. Necessary for training.+- `num_frames`: the number of frames in the video. Necessary for training.+- `fps`: the frame rate of the video (optional)+- `width`: the width of the video frame. Necessary for STDiT2.+- `height`: the height of the video frame. Necessary for STDiT2.+- `resolution`: height x width (optional)+- `aspect_ratio`: the aspect ratio of the video frame (height / width) (optional)+- `aes`: aesthetic score calculated by [asethetic scorer](/tools/aesthetic/README.md) (optional)+- `flow`: optical flow score calculated by [UniMatch](/tools/scoring/README.md) (optional)+- `match`: matching score of a image-text/video-text pair calculated with [clip](/tools/scoring/README.md) (optional)+- `cmotion`: the camera motion (optional)++Example:```csv-path, text, num_frames, fps, width, height, aspect_ratio, aesthetic_score, clip_score, ...+path, text, num_frames, fps, width, height, aspect_ratio, aes, flow, match, .../absolute/path/to/image1.jpg, caption1, num_of_frames/absolute/path/to/video2.mp4, caption2, num_of_frames```@@ -32,20 +46,6 @@ df = pd.read_csv(input_path)df = df.to_csv(output_path, index=False)```-The columns are defined as follows:--- `path`: the relative/absolute path or url to the image or video file. The only required column.-- `text`: the caption or description of the image or video. Necessary for training.-- `num_frames`: the number of frames in the video. Necessary for training.-- `fps`: the frame rate of the video. Optional.-- `width`: the width of the video frame. Necessary for STDiT2.-- `height`: the height of the video frame. Necessary for STDiT2.-- `resolution`: height x width.-- `aspect_ratio`: the aspect ratio of the video frame (height divided by width). Optional.-- `aes`: the aesthetic score by [asethetic scorer](/tools/aesthetic/README.md). Optional.-- `match`: the clip score by [clip scorer](/tools/clip/README.md). Optional.-- `cmotion`: the camera motion-## Dataset to CSVAs a start point, `convert.py` is used to convert the dataset to a CSV file. You can use the following commands to convert the dataset to a CSV file: