Open-Sora: Democratizing Efficient Video Production for All
Python
[feat] update requirements
zhengzangwb93a254abb5f
diff --git a/README.md b/README.mdindex 60a900a..3ca35ef 100644--- a/README.md+++ b/README.md@@ -146,7 +146,7 @@ Other useful documents and links are listed below.### Install from Source-For CUDA 12.1, you can install the dependencies with the following commands. Otherwise, please refer to [Installation](docs/installation.md) for more instructions.+For CUDA 12.1, you can install the dependencies with the following commands. Otherwise, please refer to [Installation](docs/installation.md) for more instructions on different cuda version, and additional dependency for data preprocessing.```bash# create a virtual env and activate (conda as an example)@@ -159,9 +159,24 @@ pip install -r requirements/requirements_cu121.txt# install this projectgit clone https://github.com/hpcaitech/Open-Soracd Open-Sora++# the default installation is for inference onlypip install -v .```+(Optional, recommended for fast speed, especially for training) To enable `layernorm_kernel` and `flash_attn`, you need to install `apex` and `flash-attn` with the following commands.++```bash+# install flash attention+# set enable_flash_attn=False in config to disable flash attention+pip install packaging ninja+pip install flash-attn --no-build-isolation++# install apex+# set enable_layernorm_kernel=False in config to disable apex+pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" git+https://github.com/NVIDIA/apex.git+```+### Use DockerRun the following command to build a docker image from Dockerfile provided.diff --git a/docs/installation.md b/docs/installation.mdindex d74b923..9c41d0d 100644--- a/docs/installation.md+++ b/docs/installation.md@@ -18,23 +18,11 @@ pip install torch==2.2.2 torchvision==0.17.2 --index-url https://download.pytorcpip install xformers --index-url https://download.pytorch.org/whl/cu121```+## Different Dependencies++The default installation is for inference only. Other optional dependencies are listed below.+```bash-# install flash attention (optional)-# set enable_flash_attn=False in config to avoid using flash attention-pip install packaging ninja-pip install flash-attn --no-build-isolation--# install apex (optional)-# set enable_layernorm_kernel=False in config to avoid using apex-pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" git+https://github.com/NVIDIA/apex.git+pip install -v .[data] # for data preprocessing+pip install -v .[eval] # for evaluation```--gdown-pre-commit-pyarrow-tensorboard-transformers-wandb-pandarallel-gradio-spacesdiff --git a/notebooks/inference.ipynb b/notebooks/inference.ipynbindex 7ef8489..170d55b 100644--- a/notebooks/inference.ipynb+++ b/notebooks/inference.ipynb@@ -16,7 +16,7 @@},{"cell_type": "code",- "execution_count": 10,+ "execution_count": null,"metadata": {},"outputs": [],"source": [@@ -38,26 +38,9 @@},{"cell_type": "code",- "execution_count": 3,+ "execution_count": null,"metadata": {},- "outputs": [- {- "name": "stderr",- "output_type": "stream",- "text": [- "/home/zhengzangwei/.conda/envs/opensora/lib/python3.10/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",- " _torch_pytree._register_pytree_node(\n",- "/home/zhengzangwei/.conda/envs/opensora/lib/python3.10/site-packages/colossalai/pipeline/schedule/_utils.py:19: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",- " _register_pytree_node(OrderedDict, _odict_flatten, _odict_unflatten)\n",- "/home/zhengzangwei/.conda/envs/opensora/lib/python3.10/site-packages/torch/utils/_pytree.py:254: UserWarning: <class 'collections.OrderedDict'> is already registered as pytree node. Overwriting the previous registration.\n",- " warnings.warn(\n",- "/home/zhengzangwei/.conda/envs/opensora/lib/python3.10/site-packages/transformers/utils/generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",- " _torch_pytree._register_pytree_node(\n",- "/home/zhengzangwei/.conda/envs/opensora/lib/python3.10/site-packages/transformers/utils/generic.py:309: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",- " _torch_pytree._register_pytree_node(\n"- ]- }- ],+ "outputs": [],"source": ["import os\n","from pprint import pformat\n",@@ -90,20 +73,9 @@},{"cell_type": "code",- "execution_count": 20,+ "execution_count": null,"metadata": {},- "outputs": [- {- "data": {- "text/plain": [- "42"- ]- },- "execution_count": 20,- "metadata": {},- "output_type": "execute_result"- }- ],+ "outputs": [],"source": ["torch.set_grad_enabled(False)\n","\n",@@ -124,24 +96,9 @@},{"cell_type": "code",- "execution_count": 5,+ "execution_count": null,"metadata": {},- "outputs": [- {- "data": {- "application/vnd.jupyter.widget-view+json": {- "model_id": "dbb26c0ba0824c69b1e1d293fbfe3d9a",- "version_major": 2,- "version_minor": 0- },- "text/plain": [- "Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]"- ]- },- "metadata": {},- "output_type": "display_data"- }- ],+ "outputs": [],"source": ["# == build text-encoder and vae ==\n","text_encoder = build_module(cfg.text_encoder, MODELS, device=device)\n",@@ -174,7 +131,7 @@},{"cell_type": "code",- "execution_count": 26,+ "execution_count": null,"metadata": {},"outputs": [],"source": [@@ -314,7 +271,7 @@},{"cell_type": "code",- "execution_count": 16,+ "execution_count": null,"metadata": {},"outputs": [],"source": [@@ -513,7 +470,7 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.10.9"+ "version": "3.10.14"}},"nbformat": 4,diff --git a/requirements/requirements-data.txt b/requirements/requirements-data.txtnew file mode 100644index 0000000..d427bea--- /dev/null+++ b/requirements/requirements-data.txt@@ -0,0 +1,4 @@+gdown++# [aesthetic]+clip @ git+https://github.com/openai/CLIP.gitdiff --git a/requirements/requirements-eval.txt b/requirements/requirements-eval.txtnew file mode 100644index 0000000..e69de29diff --git a/requirements/requirements-fast.txt b/requirements/requirements-fast.txtdeleted file mode 100644index 0c02389..0000000--- a/requirements/requirements-fast.txt+++ /dev/null@@ -1,3 +0,0 @@-packaging-ninja-flash-attn --no-build-isolationdiff --git a/requirements/requirements.txt b/requirements/requirements.txtindex e124aca..8a7b440 100644--- a/requirements/requirements.txt+++ b/requirements/requirements.txt@@ -7,3 +7,20 @@ ftfy>=6.2.0 # for t5diffusers==0.27.2 # for vaeaccelerate==0.29.2 # for t5av>=12.0.0++# [gradio]+gradio+spaces++# [notebook]+ipykernel+ipywidgets++# [training]+wandb+tensorboard+pandarallel+pyarrow # for parquet++# [dev]+pre-commitdiff --git a/setup.py b/setup.pyindex fea582c..9743539 100644--- a/setup.py+++ b/setup.py@@ -77,6 +77,7 @@ setup("Topic :: System :: Distributed Computing",],extras_require={- "fast": ["flash-attn --no-build-isolation"],+ "data": fetch_requirements("requirements/requirements-data.txt"),+ "eval": fetch_requirements("requirements/requirements-eval.txt"),},)