SOTA Open Source TTS
Update lora document
Lengyueb5a3f899eeec
diff --git a/docs/en/finetune.md b/docs/en/finetune.mdindex 7d51571..71fe057 100644--- a/docs/en/finetune.md+++ b/docs/en/finetune.md@@ -166,11 +166,14 @@ huggingface-cli download fishaudio/speech-lm-v1 text2semantic-400m-v0.2-4k.pth -Finally, you can start the fine-tuning by running the following command:```bash-python fish_speech/train.py --config-name text2semantic_finetune_spk+python fish_speech/train.py --config-name text2semantic_finetune```+!!! info+ If you want to use lora, please use `--config-name text2semantic_finetune_lora` to start fine-tuning.+!!! note- You can modify the training parameters such as `batch_size`, `gradient_accumulation_steps`, etc. to fit your GPU memory by modifying `fish_speech/configs/text2semantic_finetune_spk.yaml`.+ You can modify the training parameters such as `batch_size`, `gradient_accumulation_steps`, etc. to fit your GPU memory by modifying `fish_speech/configs/text2semantic_finetune.yaml`.After training is complete, you can refer to the [inference](inference.md) section, and use `--speaker SPK1` to generate speech.diff --git a/docs/en/index.md b/docs/en/index.mdindex de9a7a6..52c0833 100644--- a/docs/en/index.md+++ b/docs/en/index.md@@ -33,6 +33,7 @@ pip3 install -e .## Changelog+- 2023/12/28: Added `lora` fine-tuning support.- 2023/12/27: Add `gradient checkpointing`, `causual sampling`, and `flash-attn` support.- 2023/12/19: Updated webui and HTTP API.- 2023/12/18: Updated fine-tuning documentation and related examples.diff --git a/docs/en/inference.md b/docs/en/inference.mdindex db10340..4cd83ea 100644--- a/docs/en/inference.md+++ b/docs/en/inference.md@@ -51,6 +51,10 @@ This command will create a `codes_N` file in the working directory, where N is a!!! infoFor GPUs that do not support bf16, you may need to use the `--half` parameter.+!!! warning+ If you are using your own fine-tuned model, please be sure to carry the `--speaker` parameter to ensure the stability of pronunciation.+ If you are using lora, please use `--config-name text2semantic_finetune_lora` to load the model.+### 3. Generate vocals from semantic tokens:```bashpython tools/vqgan/inference.py \diff --git a/docs/zh/finetune.md b/docs/zh/finetune.mdindex f4999fe..115a978 100644--- a/docs/zh/finetune.md+++ b/docs/zh/finetune.md@@ -186,11 +186,14 @@ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/speech-lm-v最后, 你可以运行以下命令来启动微调:```bash-python fish_speech/train.py --config-name text2semantic_finetune_spk+python fish_speech/train.py --config-name text2semantic_finetune```!!! note- 你可以通过修改 `fish_speech/configs/text2semantic_finetune_spk.yaml` 来修改训练参数如 `batch_size`, `gradient_accumulation_steps` 等, 来适应你的显存.+ 如果你想使用 lora, 请使用 `--config-name text2semantic_finetune_lora` 来启动微调.++!!! note+ 你可以通过修改 `fish_speech/configs/text2semantic_finetune.yaml` 来修改训练参数如 `batch_size`, `gradient_accumulation_steps` 等, 来适应你的显存.训练结束后, 你可以参考 [推理](inference.md) 部分, 并携带 `--speaker SPK1` 参数来测试你的模型.diff --git a/docs/zh/index.md b/docs/zh/index.mdindex 010fec0..9b43b34 100644--- a/docs/zh/index.md+++ b/docs/zh/index.md@@ -33,6 +33,7 @@ pip3 install -e .## 更新日志+- 2023/12/28: 添加了 `lora` 微调支持.- 2023/12/27: 添加了 `gradient checkpointing`, `causual sampling` 和 `flash-attn` 支持.- 2023/12/19: 更新了 Webui 和 HTTP API.- 2023/12/18: 更新了微调文档和相关例子.diff --git a/docs/zh/inference.md b/docs/zh/inference.mdindex c518ac1..80b9cc0 100644--- a/docs/zh/inference.md+++ b/docs/zh/inference.md@@ -57,7 +57,8 @@ python tools/llama/generate.py \对于不支持 bf16 的 GPU, 你可能需要使用 `--half` 参数.!!! warning- 如果你在使用自己微调的模型, 请务必携带 `--speaker` 参数来保证发音的稳定性.+ 如果你在使用自己微调的模型, 请务必携带 `--speaker` 参数来保证发音的稳定性.+ 如果你使用了 lora, 请使用 `--config-name text2semantic_finetune_lora` 来加载模型.### 3. 从语义 token 生成人声:```bashdiff --git a/pyproject.toml b/pyproject.tomlindex eca2885..a688ae9 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -34,6 +34,7 @@ dependencies = ["zibai-server>=0.9.0","loguru>=0.6.0","WeTextProcessing>=0.1.10",+ "loralib>=0.1.2"][project.optional-dependencies]