Skip to content

SOTA Open Source TTS

update checkpoint path

Lengyuedad516d86df6
diff --git a/API_FLAGS.txt b/API_FLAGS.txt
index 4e8b6b7..d0be18d 100644
--- a/API_FLAGS.txt
+++ b/API_FLAGS.txt
@@ -1,6 +1,6 @@
# --infer
# --api
--listen 0.0.0.0:8080 \
---llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
---decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+--llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+--decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
diff --git a/docs/en/finetune.md b/docs/en/finetune.md
index b76e1df..72cc6ef 100644
--- a/docs/en/finetune.md
+++ b/docs/en/finetune.md
@@ -36,7 +36,7 @@ You need to convert your dataset into the above format and place it under `data`
Make sure you have downloaded the VQGAN weights. If not, run the following command:
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
You can then run the following command to extract semantic tokens:
@@ -45,7 +45,7 @@ You can then run the following command to extract semantic tokens:
python tools/vqgan/extract_vq.py data \
--num-workers 1 --batch-size 16 \
--config-name "firefly_gan_vq" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
!!! note
@@ -89,7 +89,7 @@ After the command finishes executing, you should see the `quantized-dataset-ft.p
Similarly, make sure you have downloaded the `LLAMA` weights. If not, run the following command:
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
Finally, you can start the fine-tuning by running the following command:
@@ -117,9 +117,9 @@ After training, you need to convert the LoRA weights to regular weights before p
```bash
python tools/llama/merge_lora.py \
--lora-config r_8_alpha_16 \
- --base-weight checkpoints/fish-speech-1.2-sft \
+ --base-weight checkpoints/fish-speech-1.4 \
--lora-weight results/$project/checkpoints/step_000000010.ckpt \
- --output checkpoints/fish-speech-1.2-sft-yth-lora/
+ --output checkpoints/fish-speech-1.4-yth-lora/
```
!!! note
You may also try other checkpoints. We suggest using the earliest checkpoint that meets your requirements, as they often perform better on out-of-distribution (OOD) data.
diff --git a/docs/en/inference.md b/docs/en/inference.md
index 6cd8483..da34bd7 100644
--- a/docs/en/inference.md
+++ b/docs/en/inference.md
@@ -15,7 +15,7 @@ Inference support command line, HTTP API and web UI.
Download the required `vqgan` and `llama` models from our Hugging Face repository.
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
### 1. Generate prompt from voice:
@@ -26,7 +26,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/f
```bash
python tools/vqgan/inference.py \
-i "paimon.wav" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
You should get a `fake.npy` file.
@@ -38,7 +38,7 @@ python tools/llama/generate.py \
--text "The text you want to convert" \
--prompt-text "Your reference text" \
--prompt-tokens "fake.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \
+ --checkpoint-path "checkpoints/fish-speech-1.4" \
--num-samples 2 \
--compile
```
@@ -59,7 +59,7 @@ This command will create a `codes_N` file in the working directory, where N is a
```bash
python tools/vqgan/inference.py \
-i "codes_0.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
## HTTP API Inference
@@ -69,8 +69,8 @@ We provide a HTTP API for inference. You can use the following command to start
```bash
python -m tools.api \
--listen 0.0.0.0:8080 \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
@@ -142,8 +142,8 @@ You can start the WebUI using the following command:
```bash
python -m tools.webui \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
diff --git a/docs/ja/finetune.md b/docs/ja/finetune.md
index 1549528..2a0381d 100644
--- a/docs/ja/finetune.md
+++ b/docs/ja/finetune.md
@@ -36,7 +36,7 @@
VQGANの重みをダウンロードしたことを確認してください。まだダウンロードしていない場合は、次のコマンドを実行してください。
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
次に、次のコマンドを実行してセマンティックトークンを抽出できます。
@@ -45,7 +45,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/f
python tools/vqgan/extract_vq.py data \
--num-workers 1 --batch-size 16 \
--config-name "firefly_gan_vq" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
!!! note
@@ -89,7 +89,7 @@ python tools/llama/build_dataset.py \
同様に、`LLAMA`の重みをダウンロードしたことを確認してください。まだダウンロードしていない場合は、次のコマンドを実行してください。
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
最後に、次のコマンドを実行して微調整を開始できます。
@@ -117,9 +117,9 @@ python fish_speech/train.py --config-name text2semantic_finetune \
```bash
python tools/llama/merge_lora.py \
--lora-config r_8_alpha_16 \
- --base-weight checkpoints/fish-speech-1.2-sft \
+ --base-weight checkpoints/fish-speech-1.4 \
--lora-weight results/$project/checkpoints/step_000000010.ckpt \
- --output checkpoints/fish-speech-1.2-sft-yth-lora/
+ --output checkpoints/fish-speech-1.4-yth-lora/
```
!!! note
他のチェックポイントを試すこともできます。要件を満たす最も早いチェックポイントを使用することをお勧めします。これらは通常、分布外(OOD)データでより良いパフォーマンスを発揮します。
diff --git a/docs/ja/inference.md b/docs/ja/inference.md
index 3080619..4ca5576 100644
--- a/docs/ja/inference.md
+++ b/docs/ja/inference.md
@@ -15,7 +15,7 @@
必要な`vqgan`および`llama`モデルを Hugging Face リポジトリからダウンロードします。
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
### 1. 音声からプロンプトを生成する:
@@ -26,7 +26,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/f
```bash
python tools/vqgan/inference.py \
-i "paimon.wav" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
`fake.npy`ファイルが生成されるはずです。
@@ -38,7 +38,7 @@ python tools/llama/generate.py \
--text "変換したいテキスト" \
--prompt-text "参照テキスト" \
--prompt-tokens "fake.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \
+ --checkpoint-path "checkpoints/fish-speech-1.4" \
--num-samples 2 \
--compile
```
@@ -59,7 +59,7 @@ python tools/llama/generate.py \
```bash
python tools/vqgan/inference.py \
-i "codes_0.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
## HTTP API 推論
@@ -69,8 +69,8 @@ python tools/vqgan/inference.py \
```bash
python -m tools.api \
--listen 0.0.0.0:8080 \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
@@ -146,8 +146,8 @@ python -m tools.post_api \
```bash
python -m tools.webui \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
diff --git a/docs/pt/finetune.md b/docs/pt/finetune.md
index dc7ec34..a8e52ae 100644
--- a/docs/pt/finetune.md
+++ b/docs/pt/finetune.md
@@ -36,7 +36,7 @@ Você precisa converter seu conjunto de dados para o formato acima e colocá-lo
Certifique-se de ter baixado os pesos do VQGAN. Se não, execute o seguinte comando:
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
Em seguida, você pode executar o seguinte comando para extrair os tokens semânticos:
@@ -45,7 +45,7 @@ Em seguida, você pode executar o seguinte comando para extrair os tokens semân
python tools/vqgan/extract_vq.py data \
--num-workers 1 --batch-size 16 \
--config-name "firefly_gan_vq" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
!!! note
@@ -89,7 +89,7 @@ Após executar o comando, você deverá ver o arquivo `quantized-dataset-ft.prot
Da mesma forma, certifique-se de ter baixado os pesos do `LLAMA`. Se não, execute o seguinte comando:
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
E então, execute o seguinte comando para iniciar o ajuste fino:
@@ -117,9 +117,9 @@ Após o treinamento, é preciso converter os pesos do LoRA em pesos regulares an
```bash
python tools/llama/merge_lora.py \
--lora-config r_8_alpha_16 \
- --base-weight checkpoints/fish-speech-1.2-sft \
+ --base-weight checkpoints/fish-speech-1.4 \
--lora-weight results/$project/checkpoints/step_000000010.ckpt \
- --output checkpoints/fish-speech-1.2-sft-yth-lora/
+ --output checkpoints/fish-speech-1.4-yth-lora/
```
!!! note
É possível também tentar outros checkpoints. Sugerimos usar o checkpoint que melhor atenda aos seus requisitos, pois eles geralmente têm um desempenho melhor em dados fora da distribuição (OOD).
diff --git a/docs/pt/inference.md b/docs/pt/inference.md
index 0a4e718..6a8ff5c 100644
--- a/docs/pt/inference.md
+++ b/docs/pt/inference.md
@@ -15,7 +15,7 @@ Suporte para inferência por linha de comando, API HTTP e interface web (WebUI).
Baixe os modelos `vqgan` e `llama` necessários do nosso repositório Hugging Face.
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
### 1. Gerar prompt a partir da voz:
@@ -26,7 +26,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/f
```bash
python tools/vqgan/inference.py \
-i "paimon.wav" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
Você deverá obter um arquivo `fake.npy`.
@@ -38,7 +38,7 @@ python tools/llama/generate.py \
--text "O texto que você deseja converter" \
--prompt-text "Seu texto de referência" \
--prompt-tokens "fake.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \
+ --checkpoint-path "checkpoints/fish-speech-1.4" \
--num-samples 2 \
--compile
```
@@ -59,7 +59,7 @@ Este comando criará um arquivo `codes_N` no diretório de trabalho, onde N é u
```bash
python tools/vqgan/inference.py \
-i "codes_0.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
## Inferência por API HTTP
@@ -69,8 +69,8 @@ Fornecemos uma API HTTP para inferência. O seguinte comando pode ser usado para
```bash
python -m tools.api \
--listen 0.0.0.0:8080 \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
@@ -142,8 +142,8 @@ Para iniciar a WebUI de Inferência execute o seguinte comando:
```bash
python -m tools.webui \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
diff --git a/docs/zh/finetune.md b/docs/zh/finetune.md
index 7d47e60..c72722c 100644
--- a/docs/zh/finetune.md
+++ b/docs/zh/finetune.md
@@ -34,13 +34,13 @@
确保你已经下载了 vqgan 权重, 如果没有, 请运行以下命令:
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
对于中国大陆用户, 可使用 mirror 下载.
```bash
-HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
随后可运行以下命令来提取语义 token:
@@ -49,7 +49,7 @@ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech
python tools/vqgan/extract_vq.py data \
--num-workers 1 --batch-size 16 \
--config-name "firefly_gan_vq" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
!!! note
@@ -93,13 +93,13 @@ python tools/llama/build_dataset.py \
同样的, 请确保你已经下载了 `LLAMA` 权重, 如果没有, 请运行以下命令:
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
对于中国大陆用户, 可使用 mirror 下载.
```bash
-HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
最后, 你可以运行以下命令来启动微调:
@@ -127,9 +127,9 @@ python fish_speech/train.py --config-name text2semantic_finetune \
```bash
python tools/llama/merge_lora.py \
--lora-config r_8_alpha_16 \
- --base-weight checkpoints/fish-speech-1.2-sft \
+ --base-weight checkpoints/fish-speech-1.4 \
--lora-weight results/$project/checkpoints/step_000000010.ckpt \
- --output checkpoints/fish-speech-1.2-sft-yth-lora/
+ --output checkpoints/fish-speech-1.4-yth-lora/
```
!!! note
diff --git a/docs/zh/index.md b/docs/zh/index.md
index 11324bd..db7af4c 100644
--- a/docs/zh/index.md
+++ b/docs/zh/index.md
@@ -138,13 +138,13 @@ apt install libsox-dev
确保您在 docker 容器内的终端,然后再从我们的 huggingface 仓库下载所需的 `vqgan` 和 `llama` 模型。
```bash
- huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+ huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
对于中国大陆用户,可以通过镜像站下载。
```bash
- HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
4. 配置环境变量,访问 WebUI
diff --git a/docs/zh/inference.md b/docs/zh/inference.md
index 80df182..40a2fdd 100644
--- a/docs/zh/inference.md
+++ b/docs/zh/inference.md
@@ -15,13 +15,13 @@
从我们的 huggingface 仓库下载所需的 `vqgan` 和 `llama` 模型。
```bash
-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
对于中国大陆用户,可使用 mirror 下载。
```bash
-HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft
+HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4
```
### 1. 从语音生成 prompt:
@@ -32,7 +32,7 @@ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech
```bash
python tools/vqgan/inference.py \
-i "paimon.wav" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
你应该能得到一个 `fake.npy` 文件.
@@ -44,7 +44,7 @@ python tools/llama/generate.py \
--text "要转换的文本" \
--prompt-text "你的参考文本" \
--prompt-tokens "fake.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \
+ --checkpoint-path "checkpoints/fish-speech-1.4" \
--num-samples 2 \
--compile
```
@@ -65,7 +65,7 @@ python tools/llama/generate.py \
```bash
python tools/vqgan/inference.py \
-i "codes_0.npy" \
- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
```
## HTTP API 推理
@@ -75,8 +75,8 @@ python tools/vqgan/inference.py \
```bash
python -m tools.api \
--listen 0.0.0.0:8080 \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
如果你想要加速推理,可以加上`--compile`参数。
@@ -153,8 +153,8 @@ python -m tools.post_api \
```bash
python -m tools.webui \
- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \
- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \
+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \
+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \
--decoder-config-name firefly_gan_vq
```
diff --git a/fish_speech/configs/text2semantic_finetune.yaml b/fish_speech/configs/text2semantic_finetune.yaml
index 1bf8fd6..f4c1993 100644
--- a/fish_speech/configs/text2semantic_finetune.yaml
+++ b/fish_speech/configs/text2semantic_finetune.yaml
@@ -4,7 +4,7 @@ defaults:
project: text2semantic_finetune_dual_ar
max_length: 4096
-pretrained_ckpt_path: checkpoints/fish-speech-1.2-sft
+pretrained_ckpt_path: checkpoints/fish-speech-1.4
# Lightning Trainer
trainer:
diff --git a/fish_speech/webui/manage.py b/fish_speech/webui/manage.py
index 9c183ac..09c5f00 100644
--- a/fish_speech/webui/manage.py
+++ b/fish_speech/webui/manage.py
@@ -469,7 +469,7 @@ def train_process(
"--config-name",
"firefly_gan_vq",
"--checkpoint-path",
- "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
]
)
@@ -485,7 +485,7 @@ def train_process(
"16",
]
)
- ckpt_path = "checkpoints/fish-speech-1.2-sft/model.pth"
+ ckpt_path = "checkpoints/fish-speech-1.4/model.pth"
lora_prefix = "lora_" if llama_use_lora else ""
llama_name = lora_prefix + "text2semantic_" + new_project
latest = next(
@@ -925,9 +925,9 @@ with gr.Blocks(
"Type the path or select from the dropdown"
),
choices=[
- "checkpoints/fish-speech-1.2-sft/model.pth",
+ "checkpoints/fish-speech-1.4/model.pth",
],
- value="checkpoints/fish-speech-1.2-sft/model.pth",
+ value="checkpoints/fish-speech-1.4/model.pth",
allow_custom_value=True,
interactive=True,
)
@@ -979,7 +979,7 @@ with gr.Blocks(
"Type the path or select from the dropdown"
),
choices=list_llama_models(),
- value="checkpoints/fish-speech-1.2-sft",
+ value="checkpoints/fish-speech-1.4",
allow_custom_value=True,
interactive=True,
)
@@ -1042,7 +1042,7 @@ with gr.Blocks(
"Type the path or select from the dropdown"
),
choices=list_decoder_models(),
- value="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ value="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
allow_custom_value=True,
)
infer_decoder_config = gr.Dropdown(
@@ -1060,7 +1060,7 @@ with gr.Blocks(
info=i18n(
"Type the path or select from the dropdown"
),
- value="checkpoints/fish-speech-1.2-sft",
+ value="checkpoints/fish-speech-1.4",
choices=list_llama_models(),
allow_custom_value=True,
)
diff --git a/inference.ipynb b/inference.ipynb
index c469092..e690a80 100644
--- a/inference.ipynb
+++ b/inference.ipynb
@@ -61,7 +61,7 @@
"# !set HF_ENDPOINT=https://hf-mirror.com\n",
"# !export HF_ENDPOINT=https://hf-mirror.com \n",
"\n",
- "!huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft/"
+ "!huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4/"
]
},
{
@@ -84,8 +84,8 @@
"outputs": [],
"source": [
"!python tools/webui.py \\\n",
- " --llama-checkpoint-path checkpoints/fish-speech-1.2-sft \\\n",
- " --decoder-checkpoint-path checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth \\\n",
+ " --llama-checkpoint-path checkpoints/fish-speech-1.4 \\\n",
+ " --decoder-checkpoint-path checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth \\\n",
" # --compile"
]
},
@@ -122,7 +122,7 @@
"\n",
"!python tools/vqgan/inference.py \\\n",
" -i {src_audio} \\\n",
- " --checkpoint-path \"checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth\"\n",
+ " --checkpoint-path \"checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth\"\n",
"\n",
"from IPython.display import Audio, display\n",
"audio = Audio(filename=\"fake.wav\")\n",
@@ -158,7 +158,7 @@
" --text \"hello world\" \\\n",
" --prompt-text \"The text corresponding to reference audio\" \\\n",
" --prompt-tokens \"fake.npy\" \\\n",
- " --checkpoint-path \"checkpoints/fish-speech-1.2-sft\" \\\n",
+ " --checkpoint-path \"checkpoints/fish-speech-1.4\" \\\n",
" --num-samples 2\n",
" # --compile"
]
@@ -182,7 +182,7 @@
"source": [
"!python tools/vqgan/inference.py \\\n",
" -i \"codes_0.npy\" \\\n",
- " --checkpoint-path \"checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth\"\n",
+ " --checkpoint-path \"checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth\"\n",
"\n",
"from IPython.display import Audio, display\n",
"audio = Audio(filename=\"fake.wav\")\n",
diff --git a/pyproject.toml b/pyproject.toml
index 274e9e8..344f20b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -56,4 +56,4 @@ requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
-packages = ["fish_speech"]
+packages = ["fish_speech", "tools"]
diff --git a/tools/api.py b/tools/api.py
index 945b973..e5331a2 100644
--- a/tools/api.py
+++ b/tools/api.py
@@ -345,12 +345,12 @@ def parse_args():
parser.add_argument(
"--llama-checkpoint-path",
type=str,
- default="checkpoints/fish-speech-1.2-sft",
+ default="checkpoints/fish-speech-1.4",
)
parser.add_argument(
"--decoder-checkpoint-path",
type=str,
- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
)
parser.add_argument("--decoder-config-name", type=str, default="firefly_gan_vq")
parser.add_argument("--device", type=str, default="cuda")
diff --git a/tools/download_models.py b/tools/download_models.py
index 480f3be..9e79c34 100644
--- a/tools/download_models.py
+++ b/tools/download_models.py
@@ -22,8 +22,8 @@ def check_and_download_files(repo_id, file_list, local_dir):
# 1st
-repo_id_1 = "fishaudio/fish-speech-1.2-sft"
-local_dir_1 = "./checkpoints/fish-speech-1.2-sft"
+repo_id_1 = "fishaudio/fish-speech-1.4"
+local_dir_1 = "./checkpoints/fish-speech-1.4"
files_1 = [
"model.pth",
"README.md",
@@ -31,7 +31,7 @@ files_1 = [
"tokenizer_config.json",
"tokenizer.json",
"config.json",
- "firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ "firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
]
# 3rd
diff --git a/tools/llama/generate.py b/tools/llama/generate.py
index 919fff6..52d1c3a 100644
--- a/tools/llama/generate.py
+++ b/tools/llama/generate.py
@@ -612,7 +612,7 @@ def launch_thread_safe_queue(
@click.option(
"--checkpoint-path",
type=click.Path(path_type=Path, exists=True),
- default="checkpoints/fish-speech-1.2-sft",
+ default="checkpoints/fish-speech-1.4",
)
@click.option("--device", type=str, default="cuda")
@click.option("--compile/--no-compile", default=False)
diff --git a/tools/llama/merge_lora.py b/tools/llama/merge_lora.py
index f12eece..c1bd3cb 100644
--- a/tools/llama/merge_lora.py
+++ b/tools/llama/merge_lora.py
@@ -15,7 +15,7 @@ from fish_speech.models.text2semantic.lora import get_merged_state_dict
@click.command()
@click.option("--lora-config", type=str, default="r_8_alpha_16")
[email protected]("--base-weight", type=str, default="checkpoints/fish-speech-1.2-sft")
[email protected]("--base-weight", type=str, default="checkpoints/fish-speech-1.4")
@click.option("--lora-weight", type=str, required=True)
@click.option("--output", type=str, required=True)
def merge(lora_config, base_weight, lora_weight, output):
diff --git a/tools/llama/quantize.py b/tools/llama/quantize.py
index aae32fc..e629d94 100644
--- a/tools/llama/quantize.py
+++ b/tools/llama/quantize.py
@@ -428,7 +428,7 @@ def generate_folder_name():
@click.option(
"--checkpoint-path",
type=click.Path(path_type=Path, exists=True),
- default="checkpoints/fish-speech-1.2-sft",
+ default="checkpoints/fish-speech-1.4",
)
@click.option(
"--mode", type=str, default="int8", help="type of quantization to perform"
@@ -451,7 +451,7 @@ def quantize(checkpoint_path: Path, mode: str, groupsize: int, timestamp: str) -
precision=precision,
compile=False,
)
- vq_model = "firefly-gan-vq-fsq-4x1024-42hz-generator.pth"
+ vq_model = "firefly-gan-vq-fsq-8x1024-21hz-generator.pth"
now = timestamp if timestamp != "None" else generate_folder_name()
if mode == "int8":
diff --git a/tools/vqgan/extract_vq.py b/tools/vqgan/extract_vq.py
index bc6bc40..c24eb3f 100644
--- a/tools/vqgan/extract_vq.py
+++ b/tools/vqgan/extract_vq.py
@@ -42,7 +42,7 @@ logger.add(sys.stderr, format=logger_format)
@lru_cache(maxsize=1)
def get_model(
config_name: str = "firefly_gan_vq",
- checkpoint_path: str = "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ checkpoint_path: str = "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
device: str | torch.device = "cuda",
):
with initialize(version_base="1.3", config_path="../../fish_speech/configs"):
@@ -133,7 +133,7 @@ def process_batch(files: list[Path], model) -> float:
@click.option("--config-name", default="firefly_gan_vq")
@click.option(
"--checkpoint-path",
- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
)
@click.option("--batch-size", default=64)
@click.option("--filelist", default=None, type=Path)
diff --git a/tools/vqgan/inference.py b/tools/vqgan/inference.py
index 17c9034..000b5a4 100644
--- a/tools/vqgan/inference.py
+++ b/tools/vqgan/inference.py
@@ -59,7 +59,7 @@ def load_model(config_name, checkpoint_path, device="cuda"):
@click.option("--config-name", default="firefly_gan_vq")
@click.option(
"--checkpoint-path",
- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
)
@click.option(
"--device",
diff --git a/tools/webui.py b/tools/webui.py
index da19f6b..0aef2d6 100644
--- a/tools/webui.py
+++ b/tools/webui.py
@@ -560,12 +560,12 @@ def parse_args():
parser.add_argument(
"--llama-checkpoint-path",
type=Path,
- default="checkpoints/fish-speech-1.2-sft",
+ default="checkpoints/fish-speech-1.4",
)
parser.add_argument(
"--decoder-checkpoint-path",
type=Path,
- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",
+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",
)
parser.add_argument("--decoder-config-name", type=str, default="firefly_gan_vq")
parser.add_argument("--device", type=str, default="cuda")