Skip to content

Open-Sora: Democratizing Efficient Video Production for All

Python

[feat] add lancher for eval loss and search bs

zhengzangw5160fc0fd916
diff --git a/configs/opensora-v1-2/misc/bs.py b/configs/opensora-v1-2/misc/bs.py
index c1a60cd..18fbfe6 100644
--- a/configs/opensora-v1-2/misc/bs.py
+++ b/configs/opensora-v1-2/misc/bs.py
@@ -26,13 +26,13 @@ bucket_config = {
408: (2, 8),
},
# ---
- # "512": {
- # 1: (141, 0),
- # 51: (8, 0),
- # 102: (4, 0),
- # 204: (2, 0),
- # 408: (1, 0),
- # },
+ "512": {
+ 1: (141, 0),
+ 51: (8, 0),
+ 102: (4, 0),
+ 204: (2, 0),
+ 408: (1, 0),
+ },
# ---
"480p": {
1: (50, 50),
diff --git a/eval/README.md b/eval/README.md
index 16ac166..891fd87 100644
--- a/eval/README.md
+++ b/eval/README.md
@@ -15,12 +15,23 @@ bash eval/launch.sh /path/to/ckpt
## Rectified Flow Loss
+Evaluate the rectified flow loss with the following commands.
+
```bash
-CUDA_VISIBLE_DEVICES=2 torchrun --standalone --nproc_per_node 1 scripts/misc/eval_loss.py configs/opensora-v1-2/misc/eval_loss.py --data-path /mnt/nfs-207/sora_data/meta/img_1k.csv --ckpt-path /home/lishenggui/projects/sora/Open-Sora-dev/outputs/207-STDiT3-XL-2/epoch0-global_step9000/
+# image
+torchrun --standalone --nproc_per_node 1 eval/loss/eval_loss.py configs/opensora-v1-2/misc/eval_loss.py --data-path /path/to/img.csv --ckpt-path /path/to/ckpt
+
+# video
+torchrun --standalone --nproc_per_node 1 eval/loss/eval_loss.py configs/opensora-v1-2/misc/eval_loss.py --data-path /path/to/vid.csv --ckpt-path /path/to/ckpt
-CUDA_VISIBLE_DEVICES=3 torchrun --standalone --nproc_per_node 1 scripts/misc/eval_loss.py configs/opensora-v1-2/misc/eval_loss.py --data-path /mnt/nfs-207/sora_data/meta/vid_100.csv --ckpt-path /home/lishenggui/projects/sora/Open-Sora-dev/outputs/207-STDiT3-XL-2/epoch0-global_step9000/
+# select resolution
+torchrun --standalone --nproc_per_node 1 eval/loss/eval_loss.py configs/opensora-v1-2/misc/eval_loss.py --data-path /path/to/vid.csv --ckpt-path /path/to/ckpt --resolution 720p
+```
-CUDA_VISIBLE_DEVICES=3 torchrun --standalone --nproc_per_node 1 scripts/misc/eval_loss.py configs/opensora-v1-2/misc/eval_loss.py --data-path /mnt/nfs-207/sora_data/meta/vid_100.csv --ckpt-path /home/lishenggui/projects/sora/Open-Sora-dev/outputs/207-STDiT3-XL-2/epoch0-global_step9000/ --resolution 720p
+To launch multiple jobs at once, use the following script.
+
+```bash
+bash eval/loss/launch.sh /path/to/ckpt
```
## VBench
diff --git a/scripts/misc/generate.sh b/eval/human_eval/generate.sh
similarity index 100%
rename from scripts/misc/generate.sh
rename to eval/human_eval/generate.sh
diff --git a/scripts/misc/eval_loss.py b/eval/loss/eval_loss.py
similarity index 100%
rename from scripts/misc/eval_loss.py
rename to eval/loss/eval_loss.py
diff --git a/eval/loss/launch.sh b/eval/loss/launch.sh
new file mode 100644
index 0000000..ac55f1d
--- /dev/null
+++ b/eval/loss/launch.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+set -x
+set -e
+
+CMD="torchrun --standalone --nproc_per_node 1 eval/loss/eval_loss.py configs/opensora-v1-2/misc/eval_loss.py"
+CKPT_PATH=$1
+IMG_PATH="/mnt/nfs-207/sora_data/meta/img_1k.csv"
+VID_PATH="/mnt/nfs-207/sora_data/meta/vid_100.csv"
+
+if [[ $CKPT_PATH == *"ema"* ]]; then
+ parentdir=$(dirname $CKPT_PATH)
+ CKPT_BASE=$(basename $parentdir)_ema
+else
+ CKPT_BASE=$(basename $CKPT_PATH)
+fi
+LOG_BASE=logs/loss/${CKPT_BASE}
+mkdir -p logs/loss
+echo "Logging to $LOG_BASE"
+
+CUDA_VISIBLE_DEVICES=0 $CMD --data-path $IMG_PATH --ckpt-path $CKPT_PATH >${LOG_BASE}_img.log 2>&1 &
+CUDA_VISIBLE_DEVICES=2 $CMD --data-path $VID_PATH --ckpt-path $CKPT_PATH --resolution 144p >${LOG_BASE}_144p_vid.log 2>&1 &
+CUDA_VISIBLE_DEVICES=3 $CMD --data-path $VID_PATH --ckpt-path $CKPT_PATH --resolution 240p >${LOG_BASE}_240p_vid.log 2>&1 &
+CUDA_VISIBLE_DEVICES=4 $CMD --data-path $VID_PATH --ckpt-path $CKPT_PATH --resolution 360p >${LOG_BASE}_360p_vid.log 2>&1 &
+CUDA_VISIBLE_DEVICES=5 $CMD --data-path $VID_PATH --ckpt-path $CKPT_PATH --resolution 480p >${LOG_BASE}_480p_vid.log 2>&1 &
+CUDA_VISIBLE_DEVICES=6 $CMD --data-path $VID_PATH --ckpt-path $CKPT_PATH --resolution 720p >${LOG_BASE}_720p_vid.log 2>&1 &
diff --git a/scripts/misc/launch_search_bs.sh b/scripts/misc/launch_search_bs.sh
new file mode 100644
index 0000000..fdf777f
--- /dev/null
+++ b/scripts/misc/launch_search_bs.sh
@@ -0,0 +1,19 @@
+#!/bin/bash
+
+set -x
+set -e
+
+CMD="torchrun --standalone --nproc_per_node 1 scripts/misc/search_bs.py configs/opensora-v1-2/misc/bs.py"
+DATA_PATH="/mnt/nfs-207/sora_data/meta/searchbs.csv"
+
+LOG_BASE=logs/search_bs
+mkdir -p logs/search_bs
+echo "Logging to $LOG_BASE"
+
+CUDA_VISIBLE_DEVICES=0 $CMD --data-path $DATA_PATH --resolution 144p >${LOG_BASE}/144p.log 2>&1 &
+CUDA_VISIBLE_DEVICES=1 $CMD --data-path $DATA_PATH --resolution 240p >${LOG_BASE}/240p.log 2>&1 &
+CUDA_VISIBLE_DEVICES=2 $CMD --data-path $DATA_PATH --resolution 512 >${LOG_BASE}/512.log 2>&1 &
+CUDA_VISIBLE_DEVICES=3 $CMD --data-path $DATA_PATH --resolution 480p >${LOG_BASE}/480p.log 2>&1 &
+CUDA_VISIBLE_DEVICES=4 $CMD --data-path $DATA_PATH --resolution 1024 >${LOG_BASE}/1024.log 2>&1 &
+CUDA_VISIBLE_DEVICES=5 $CMD --data-path $DATA_PATH --resolution 1080p >${LOG_BASE}/1080p.log 2>&1 &
+CUDA_VISIBLE_DEVICES=6 $CMD --data-path $DATA_PATH --resolution 2048 >${LOG_BASE}/2048.log 2>&1 &