Open-Sora: Democratizing Efficient Video Production for All
Python
format
Shen-Chenhui69791fbab4f7
diff --git a/docs/installation.md b/docs/installation.mdindex 5bd2900..df6de30 100644--- a/docs/installation.md+++ b/docs/installation.md@@ -70,6 +70,7 @@ You need to manually install LLaVA with the following command:```bashpip install --no-deps llava@git+https://github.com/haotian-liu/[email protected]```+### PLLaVA CaptioningYou need to manually install PLLaVa with the following commands:@@ -92,6 +93,7 @@ pip install flash-attn --no-build-isolationpip install -r ../../../../requirements/requirements-pllava.txt```+### Frame Interpolation```bashconda install -c conda-forge opencv@@ -104,14 +106,36 @@ pip install scenedetect[opencv] --upgrade```### OCR-We need to manualy create new environment for torch==2.0.1 in order to install [MMOCR](https://mmocr.readthedocs.io/en/dev-1.x/get_started/install.html).-For reference, we install packages of these versions.-```-torch==2.0.1-mmcv==2.0.1-mmdet==3.1.0-mmocr==1.0.1-```++You need to go into `path_to_your_env/lib/python3.10/site-packages/mmdet/__init__.py`+and change the assert of `mmcv_version < digit_version(mmcv_maximum_version)` to `mmcv_version <= digit_version(mmcv_maximum_version)`.++If you are unsure of your path to the mmdet init file, simply run our [OCR command](../tools/scoring/README.md), wait for the mmdeet assertion error on mmcv versions.+The error will contain the exact path to the mmdet init file.+++<!-- We need to manualy create new environment for torch==2.0.1 in order to install [MMOCR](https://mmocr.readthedocs.io/en/dev-1.x/get_started/install.html).+This is because the current latest MMOCR version (1.0.1) is not compatible with higher versions of mmcv that work with higher torch versions.++```bash+conda create -n ocr python=3.8+conda activate ocr+nvcc --version # Check that you have CUDA 11.8 or 11.7, if not install CUDA first+pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118+pip install packaging==24.0+pip install openmim==0.3.9+mim install mmengine==0.10.4+mim install mmcv==2.0.1+mim install mmdet==3.1.0+mim install mmocr==1.0.1+pip install colossalai==0.3.6++# install apex+pip install -U pip+pip install -U wheel+pip install setuptools==60.2.0+pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" git+https://github.com/NVIDIA/apex.git+``` -->diff --git a/requirements/requirements-data.txt b/requirements/requirements-data.txtindex 01c6736..8fabae1 100644--- a/requirements/requirements-data.txt+++ b/requirements/requirements-data.txt@@ -45,3 +45,9 @@ imageio-ffmpeg==0.4.9# [aesthetic]clip @ git+https://github.com/openai/CLIP.gitdecord==0.6.0++# [ocr]+mmcv==2.1.0+mmdet==3.1.0+mmocr==1.0.1+detectron2 @ git+https://github.com/facebookresearch/detectron2.git@ff53992diff --git a/requirements/requirements-eval.txt b/requirements/requirements-eval.txtindex be9bff5..4a1293c 100644--- a/requirements/requirements-eval.txt+++ b/requirements/requirements-eval.txt@@ -1,5 +1,6 @@# [vbench]-detectron2 @ git+https://github.com/facebookresearch/[email protected]+detectron2 @ git+https://github.com/facebookresearch/detectron2.git@ff53992+# [vae]diff --git a/tools/scoring/README.md b/tools/scoring/README.mdindex bdf74c7..40949c7 100644--- a/tools/scoring/README.md+++ b/tools/scoring/README.md@@ -64,7 +64,7 @@ Some videos are of dense text scenes like news broadcast and advertisement, whicWe apply Optical Character Recognition (OCR) to detect texts and drop samples with dense texts. Here, we usethe [DBNet++](https://arxiv.org/abs/2202.10304) model implemented by [MMOCR](https://github.com/open-mmlab/mmocr/).-First, install the required packages following our [installation instructions](../../docs/installation.md)'s "OCR" section.+First, install the required packages following our [installation instructions](../../docs/installation.md)'s "Data Dependencies" and "OCR" section.<!-- First, install [MMOCR](https://mmocr.readthedocs.io/en/dev-1.x/get_started/install.html).For reference, we install packages of these versions.@@ -76,8 +76,11 @@ mmocr==1.0.1``` -->Then, run the following command. **Make sure** the meta file has column `path` (path to the sample).-```bash+<!-- ```bashtorchrun --standalone --nproc_per_node 8 tools/scoring/ocr/inference.py /path/to/meta.csv+``` -->+```bash+torchrun --standalone --nproc_per_node 8 -m tools.scoring.ocr.inference /path/to/meta.csv```This should output `/path/to/meta_ocr.csv` with column `ocr`, indicating the number of text regions with detection confidence > 0.3.