SOTA Open Source TTS
update checkpoint path
Lengyuedad516d86df6
diff --git a/API_FLAGS.txt b/API_FLAGS.txtindex 4e8b6b7..d0be18d 100644--- a/API_FLAGS.txt+++ b/API_FLAGS.txt@@ -1,6 +1,6 @@# --infer# --api--listen 0.0.0.0:8080 \---llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \---decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+--llama-checkpoint-path "checkpoints/fish-speech-1.4" \+--decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vqdiff --git a/docs/en/finetune.md b/docs/en/finetune.mdindex b76e1df..72cc6ef 100644--- a/docs/en/finetune.md+++ b/docs/en/finetune.md@@ -36,7 +36,7 @@ You need to convert your dataset into the above format and place it under `data`Make sure you have downloaded the VQGAN weights. If not, run the following command:```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```You can then run the following command to extract semantic tokens:@@ -45,7 +45,7 @@ You can then run the following command to extract semantic tokens:python tools/vqgan/extract_vq.py data \--num-workers 1 --batch-size 16 \--config-name "firefly_gan_vq" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```!!! note@@ -89,7 +89,7 @@ After the command finishes executing, you should see the `quantized-dataset-ft.pSimilarly, make sure you have downloaded the `LLAMA` weights. If not, run the following command:```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```Finally, you can start the fine-tuning by running the following command:@@ -117,9 +117,9 @@ After training, you need to convert the LoRA weights to regular weights before p```bashpython tools/llama/merge_lora.py \--lora-config r_8_alpha_16 \- --base-weight checkpoints/fish-speech-1.2-sft \+ --base-weight checkpoints/fish-speech-1.4 \--lora-weight results/$project/checkpoints/step_000000010.ckpt \- --output checkpoints/fish-speech-1.2-sft-yth-lora/+ --output checkpoints/fish-speech-1.4-yth-lora/```!!! noteYou may also try other checkpoints. We suggest using the earliest checkpoint that meets your requirements, as they often perform better on out-of-distribution (OOD) data.diff --git a/docs/en/inference.md b/docs/en/inference.mdindex 6cd8483..da34bd7 100644--- a/docs/en/inference.md+++ b/docs/en/inference.md@@ -15,7 +15,7 @@ Inference support command line, HTTP API and web UI.Download the required `vqgan` and `llama` models from our Hugging Face repository.```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```### 1. Generate prompt from voice:@@ -26,7 +26,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/f```bashpython tools/vqgan/inference.py \-i "paimon.wav" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```You should get a `fake.npy` file.@@ -38,7 +38,7 @@ python tools/llama/generate.py \--text "The text you want to convert" \--prompt-text "Your reference text" \--prompt-tokens "fake.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \+ --checkpoint-path "checkpoints/fish-speech-1.4" \--num-samples 2 \--compile```@@ -59,7 +59,7 @@ This command will create a `codes_N` file in the working directory, where N is a```bashpython tools/vqgan/inference.py \-i "codes_0.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```## HTTP API Inference@@ -69,8 +69,8 @@ We provide a HTTP API for inference. You can use the following command to start```bashpython -m tools.api \--listen 0.0.0.0:8080 \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```@@ -142,8 +142,8 @@ You can start the WebUI using the following command:```bashpython -m tools.webui \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```diff --git a/docs/ja/finetune.md b/docs/ja/finetune.mdindex 1549528..2a0381d 100644--- a/docs/ja/finetune.md+++ b/docs/ja/finetune.md@@ -36,7 +36,7 @@VQGANの重みをダウンロードしたことを確認してください。まだダウンロードしていない場合は、次のコマンドを実行してください。```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```次に、次のコマンドを実行してセマンティックトークンを抽出できます。@@ -45,7 +45,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fpython tools/vqgan/extract_vq.py data \--num-workers 1 --batch-size 16 \--config-name "firefly_gan_vq" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```!!! note@@ -89,7 +89,7 @@ python tools/llama/build_dataset.py \同様に、`LLAMA`の重みをダウンロードしたことを確認してください。まだダウンロードしていない場合は、次のコマンドを実行してください。```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```最後に、次のコマンドを実行して微調整を開始できます。@@ -117,9 +117,9 @@ python fish_speech/train.py --config-name text2semantic_finetune \```bashpython tools/llama/merge_lora.py \--lora-config r_8_alpha_16 \- --base-weight checkpoints/fish-speech-1.2-sft \+ --base-weight checkpoints/fish-speech-1.4 \--lora-weight results/$project/checkpoints/step_000000010.ckpt \- --output checkpoints/fish-speech-1.2-sft-yth-lora/+ --output checkpoints/fish-speech-1.4-yth-lora/```!!! note他のチェックポイントを試すこともできます。要件を満たす最も早いチェックポイントを使用することをお勧めします。これらは通常、分布外(OOD)データでより良いパフォーマンスを発揮します。diff --git a/docs/ja/inference.md b/docs/ja/inference.mdindex 3080619..4ca5576 100644--- a/docs/ja/inference.md+++ b/docs/ja/inference.md@@ -15,7 +15,7 @@必要な`vqgan`および`llama`モデルを Hugging Face リポジトリからダウンロードします。```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```### 1. 音声からプロンプトを生成する:@@ -26,7 +26,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/f```bashpython tools/vqgan/inference.py \-i "paimon.wav" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"````fake.npy`ファイルが生成されるはずです。@@ -38,7 +38,7 @@ python tools/llama/generate.py \--text "変換したいテキスト" \--prompt-text "参照テキスト" \--prompt-tokens "fake.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \+ --checkpoint-path "checkpoints/fish-speech-1.4" \--num-samples 2 \--compile```@@ -59,7 +59,7 @@ python tools/llama/generate.py \```bashpython tools/vqgan/inference.py \-i "codes_0.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```## HTTP API 推論@@ -69,8 +69,8 @@ python tools/vqgan/inference.py \```bashpython -m tools.api \--listen 0.0.0.0:8080 \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```@@ -146,8 +146,8 @@ python -m tools.post_api \```bashpython -m tools.webui \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```diff --git a/docs/pt/finetune.md b/docs/pt/finetune.mdindex dc7ec34..a8e52ae 100644--- a/docs/pt/finetune.md+++ b/docs/pt/finetune.md@@ -36,7 +36,7 @@ Você precisa converter seu conjunto de dados para o formato acima e colocá-loCertifique-se de ter baixado os pesos do VQGAN. Se não, execute o seguinte comando:```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```Em seguida, você pode executar o seguinte comando para extrair os tokens semânticos:@@ -45,7 +45,7 @@ Em seguida, você pode executar o seguinte comando para extrair os tokens semânpython tools/vqgan/extract_vq.py data \--num-workers 1 --batch-size 16 \--config-name "firefly_gan_vq" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```!!! note@@ -89,7 +89,7 @@ Após executar o comando, você deverá ver o arquivo `quantized-dataset-ft.protDa mesma forma, certifique-se de ter baixado os pesos do `LLAMA`. Se não, execute o seguinte comando:```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```E então, execute o seguinte comando para iniciar o ajuste fino:@@ -117,9 +117,9 @@ Após o treinamento, é preciso converter os pesos do LoRA em pesos regulares an```bashpython tools/llama/merge_lora.py \--lora-config r_8_alpha_16 \- --base-weight checkpoints/fish-speech-1.2-sft \+ --base-weight checkpoints/fish-speech-1.4 \--lora-weight results/$project/checkpoints/step_000000010.ckpt \- --output checkpoints/fish-speech-1.2-sft-yth-lora/+ --output checkpoints/fish-speech-1.4-yth-lora/```!!! noteÉ possível também tentar outros checkpoints. Sugerimos usar o checkpoint que melhor atenda aos seus requisitos, pois eles geralmente têm um desempenho melhor em dados fora da distribuição (OOD).diff --git a/docs/pt/inference.md b/docs/pt/inference.mdindex 0a4e718..6a8ff5c 100644--- a/docs/pt/inference.md+++ b/docs/pt/inference.md@@ -15,7 +15,7 @@ Suporte para inferência por linha de comando, API HTTP e interface web (WebUI).Baixe os modelos `vqgan` e `llama` necessários do nosso repositório Hugging Face.```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```### 1. Gerar prompt a partir da voz:@@ -26,7 +26,7 @@ huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/f```bashpython tools/vqgan/inference.py \-i "paimon.wav" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```Você deverá obter um arquivo `fake.npy`.@@ -38,7 +38,7 @@ python tools/llama/generate.py \--text "O texto que você deseja converter" \--prompt-text "Seu texto de referência" \--prompt-tokens "fake.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \+ --checkpoint-path "checkpoints/fish-speech-1.4" \--num-samples 2 \--compile```@@ -59,7 +59,7 @@ Este comando criará um arquivo `codes_N` no diretório de trabalho, onde N é u```bashpython tools/vqgan/inference.py \-i "codes_0.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```## Inferência por API HTTP@@ -69,8 +69,8 @@ Fornecemos uma API HTTP para inferência. O seguinte comando pode ser usado para```bashpython -m tools.api \--listen 0.0.0.0:8080 \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```@@ -142,8 +142,8 @@ Para iniciar a WebUI de Inferência execute o seguinte comando:```bashpython -m tools.webui \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```diff --git a/docs/zh/finetune.md b/docs/zh/finetune.mdindex 7d47e60..c72722c 100644--- a/docs/zh/finetune.md+++ b/docs/zh/finetune.md@@ -34,13 +34,13 @@确保你已经下载了 vqgan 权重, 如果没有, 请运行以下命令:```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```对于中国大陆用户, 可使用 mirror 下载.```bash-HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```随后可运行以下命令来提取语义 token:@@ -49,7 +49,7 @@ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speechpython tools/vqgan/extract_vq.py data \--num-workers 1 --batch-size 16 \--config-name "firefly_gan_vq" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```!!! note@@ -93,13 +93,13 @@ python tools/llama/build_dataset.py \同样的, 请确保你已经下载了 `LLAMA` 权重, 如果没有, 请运行以下命令:```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```对于中国大陆用户, 可使用 mirror 下载.```bash-HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```最后, 你可以运行以下命令来启动微调:@@ -127,9 +127,9 @@ python fish_speech/train.py --config-name text2semantic_finetune \```bashpython tools/llama/merge_lora.py \--lora-config r_8_alpha_16 \- --base-weight checkpoints/fish-speech-1.2-sft \+ --base-weight checkpoints/fish-speech-1.4 \--lora-weight results/$project/checkpoints/step_000000010.ckpt \- --output checkpoints/fish-speech-1.2-sft-yth-lora/+ --output checkpoints/fish-speech-1.4-yth-lora/```!!! notediff --git a/docs/zh/index.md b/docs/zh/index.mdindex 11324bd..db7af4c 100644--- a/docs/zh/index.md+++ b/docs/zh/index.md@@ -138,13 +138,13 @@ apt install libsox-dev确保您在 docker 容器内的终端,然后再从我们的 huggingface 仓库下载所需的 `vqgan` 和 `llama` 模型。```bash- huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+ huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```对于中国大陆用户,可以通过镜像站下载。```bash- HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```4. 配置环境变量,访问 WebUIdiff --git a/docs/zh/inference.md b/docs/zh/inference.mdindex 80df182..40a2fdd 100644--- a/docs/zh/inference.md+++ b/docs/zh/inference.md@@ -15,13 +15,13 @@从我们的 huggingface 仓库下载所需的 `vqgan` 和 `llama` 模型。```bash-huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```对于中国大陆用户,可使用 mirror 下载。```bash-HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft+HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4```### 1. 从语音生成 prompt:@@ -32,7 +32,7 @@ HF_ENDPOINT=https://hf-mirror.com huggingface-cli download fishaudio/fish-speech```bashpython tools/vqgan/inference.py \-i "paimon.wav" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```你应该能得到一个 `fake.npy` 文件.@@ -44,7 +44,7 @@ python tools/llama/generate.py \--text "要转换的文本" \--prompt-text "你的参考文本" \--prompt-tokens "fake.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft" \+ --checkpoint-path "checkpoints/fish-speech-1.4" \--num-samples 2 \--compile```@@ -65,7 +65,7 @@ python tools/llama/generate.py \```bashpython tools/vqgan/inference.py \-i "codes_0.npy" \- --checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ --checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth"```## HTTP API 推理@@ -75,8 +75,8 @@ python tools/vqgan/inference.py \```bashpython -m tools.api \--listen 0.0.0.0:8080 \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```如果你想要加速推理,可以加上`--compile`参数。@@ -153,8 +153,8 @@ python -m tools.post_api \```bashpython -m tools.webui \- --llama-checkpoint-path "checkpoints/fish-speech-1.2-sft" \- --decoder-checkpoint-path "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" \+ --llama-checkpoint-path "checkpoints/fish-speech-1.4" \+ --decoder-checkpoint-path "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth" \--decoder-config-name firefly_gan_vq```diff --git a/fish_speech/configs/text2semantic_finetune.yaml b/fish_speech/configs/text2semantic_finetune.yamlindex 1bf8fd6..f4c1993 100644--- a/fish_speech/configs/text2semantic_finetune.yaml+++ b/fish_speech/configs/text2semantic_finetune.yaml@@ -4,7 +4,7 @@ defaults:project: text2semantic_finetune_dual_armax_length: 4096-pretrained_ckpt_path: checkpoints/fish-speech-1.2-sft+pretrained_ckpt_path: checkpoints/fish-speech-1.4# Lightning Trainertrainer:diff --git a/fish_speech/webui/manage.py b/fish_speech/webui/manage.pyindex 9c183ac..09c5f00 100644--- a/fish_speech/webui/manage.py+++ b/fish_speech/webui/manage.py@@ -469,7 +469,7 @@ def train_process("--config-name","firefly_gan_vq","--checkpoint-path",- "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",])@@ -485,7 +485,7 @@ def train_process("16",])- ckpt_path = "checkpoints/fish-speech-1.2-sft/model.pth"+ ckpt_path = "checkpoints/fish-speech-1.4/model.pth"lora_prefix = "lora_" if llama_use_lora else ""llama_name = lora_prefix + "text2semantic_" + new_projectlatest = next(@@ -925,9 +925,9 @@ with gr.Blocks("Type the path or select from the dropdown"),choices=[- "checkpoints/fish-speech-1.2-sft/model.pth",+ "checkpoints/fish-speech-1.4/model.pth",],- value="checkpoints/fish-speech-1.2-sft/model.pth",+ value="checkpoints/fish-speech-1.4/model.pth",allow_custom_value=True,interactive=True,)@@ -979,7 +979,7 @@ with gr.Blocks("Type the path or select from the dropdown"),choices=list_llama_models(),- value="checkpoints/fish-speech-1.2-sft",+ value="checkpoints/fish-speech-1.4",allow_custom_value=True,interactive=True,)@@ -1042,7 +1042,7 @@ with gr.Blocks("Type the path or select from the dropdown"),choices=list_decoder_models(),- value="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ value="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",allow_custom_value=True,)infer_decoder_config = gr.Dropdown(@@ -1060,7 +1060,7 @@ with gr.Blocks(info=i18n("Type the path or select from the dropdown"),- value="checkpoints/fish-speech-1.2-sft",+ value="checkpoints/fish-speech-1.4",choices=list_llama_models(),allow_custom_value=True,)diff --git a/inference.ipynb b/inference.ipynbindex c469092..e690a80 100644--- a/inference.ipynb+++ b/inference.ipynb@@ -61,7 +61,7 @@"# !set HF_ENDPOINT=https://hf-mirror.com\n","# !export HF_ENDPOINT=https://hf-mirror.com \n","\n",- "!huggingface-cli download fishaudio/fish-speech-1.2-sft --local-dir checkpoints/fish-speech-1.2-sft/"+ "!huggingface-cli download fishaudio/fish-speech-1.4 --local-dir checkpoints/fish-speech-1.4/"]},{@@ -84,8 +84,8 @@"outputs": [],"source": ["!python tools/webui.py \\\n",- " --llama-checkpoint-path checkpoints/fish-speech-1.2-sft \\\n",- " --decoder-checkpoint-path checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth \\\n",+ " --llama-checkpoint-path checkpoints/fish-speech-1.4 \\\n",+ " --decoder-checkpoint-path checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth \\\n"," # --compile"]},@@ -122,7 +122,7 @@"\n","!python tools/vqgan/inference.py \\\n"," -i {src_audio} \\\n",- " --checkpoint-path \"checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth\"\n",+ " --checkpoint-path \"checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth\"\n","\n","from IPython.display import Audio, display\n","audio = Audio(filename=\"fake.wav\")\n",@@ -158,7 +158,7 @@" --text \"hello world\" \\\n"," --prompt-text \"The text corresponding to reference audio\" \\\n"," --prompt-tokens \"fake.npy\" \\\n",- " --checkpoint-path \"checkpoints/fish-speech-1.2-sft\" \\\n",+ " --checkpoint-path \"checkpoints/fish-speech-1.4\" \\\n"," --num-samples 2\n"," # --compile"]@@ -182,7 +182,7 @@"source": ["!python tools/vqgan/inference.py \\\n"," -i \"codes_0.npy\" \\\n",- " --checkpoint-path \"checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth\"\n",+ " --checkpoint-path \"checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth\"\n","\n","from IPython.display import Audio, display\n","audio = Audio(filename=\"fake.wav\")\n",diff --git a/pyproject.toml b/pyproject.tomlindex 274e9e8..344f20b 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -56,4 +56,4 @@ requires = ["setuptools", "setuptools-scm"]build-backend = "setuptools.build_meta"[tool.setuptools]-packages = ["fish_speech"]+packages = ["fish_speech", "tools"]diff --git a/tools/api.py b/tools/api.pyindex 945b973..e5331a2 100644--- a/tools/api.py+++ b/tools/api.py@@ -345,12 +345,12 @@ def parse_args():parser.add_argument("--llama-checkpoint-path",type=str,- default="checkpoints/fish-speech-1.2-sft",+ default="checkpoints/fish-speech-1.4",)parser.add_argument("--decoder-checkpoint-path",type=str,- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",)parser.add_argument("--decoder-config-name", type=str, default="firefly_gan_vq")parser.add_argument("--device", type=str, default="cuda")diff --git a/tools/download_models.py b/tools/download_models.pyindex 480f3be..9e79c34 100644--- a/tools/download_models.py+++ b/tools/download_models.py@@ -22,8 +22,8 @@ def check_and_download_files(repo_id, file_list, local_dir):# 1st-repo_id_1 = "fishaudio/fish-speech-1.2-sft"-local_dir_1 = "./checkpoints/fish-speech-1.2-sft"+repo_id_1 = "fishaudio/fish-speech-1.4"+local_dir_1 = "./checkpoints/fish-speech-1.4"files_1 = ["model.pth","README.md",@@ -31,7 +31,7 @@ files_1 = ["tokenizer_config.json","tokenizer.json","config.json",- "firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ "firefly-gan-vq-fsq-8x1024-21hz-generator.pth",]# 3rddiff --git a/tools/llama/generate.py b/tools/llama/generate.pyindex 919fff6..52d1c3a 100644--- a/tools/llama/generate.py+++ b/tools/llama/generate.py@@ -612,7 +612,7 @@ def launch_thread_safe_queue(@click.option("--checkpoint-path",type=click.Path(path_type=Path, exists=True),- default="checkpoints/fish-speech-1.2-sft",+ default="checkpoints/fish-speech-1.4",)@click.option("--device", type=str, default="cuda")@click.option("--compile/--no-compile", default=False)diff --git a/tools/llama/merge_lora.py b/tools/llama/merge_lora.pyindex f12eece..c1bd3cb 100644--- a/tools/llama/merge_lora.py+++ b/tools/llama/merge_lora.py@@ -15,7 +15,7 @@ from fish_speech.models.text2semantic.lora import get_merged_state_dict@click.command()@click.option("--lora-config", type=str, default="r_8_alpha_16")[email protected]("--base-weight", type=str, default="checkpoints/fish-speech-1.2-sft")[email protected]("--base-weight", type=str, default="checkpoints/fish-speech-1.4")@click.option("--lora-weight", type=str, required=True)@click.option("--output", type=str, required=True)def merge(lora_config, base_weight, lora_weight, output):diff --git a/tools/llama/quantize.py b/tools/llama/quantize.pyindex aae32fc..e629d94 100644--- a/tools/llama/quantize.py+++ b/tools/llama/quantize.py@@ -428,7 +428,7 @@ def generate_folder_name():@click.option("--checkpoint-path",type=click.Path(path_type=Path, exists=True),- default="checkpoints/fish-speech-1.2-sft",+ default="checkpoints/fish-speech-1.4",)@click.option("--mode", type=str, default="int8", help="type of quantization to perform"@@ -451,7 +451,7 @@ def quantize(checkpoint_path: Path, mode: str, groupsize: int, timestamp: str) -precision=precision,compile=False,)- vq_model = "firefly-gan-vq-fsq-4x1024-42hz-generator.pth"+ vq_model = "firefly-gan-vq-fsq-8x1024-21hz-generator.pth"now = timestamp if timestamp != "None" else generate_folder_name()if mode == "int8":diff --git a/tools/vqgan/extract_vq.py b/tools/vqgan/extract_vq.pyindex bc6bc40..c24eb3f 100644--- a/tools/vqgan/extract_vq.py+++ b/tools/vqgan/extract_vq.py@@ -42,7 +42,7 @@ logger.add(sys.stderr, format=logger_format)@lru_cache(maxsize=1)def get_model(config_name: str = "firefly_gan_vq",- checkpoint_path: str = "checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ checkpoint_path: str = "checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",device: str | torch.device = "cuda",):with initialize(version_base="1.3", config_path="../../fish_speech/configs"):@@ -133,7 +133,7 @@ def process_batch(files: list[Path], model) -> float:@click.option("--config-name", default="firefly_gan_vq")@click.option("--checkpoint-path",- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",)@click.option("--batch-size", default=64)@click.option("--filelist", default=None, type=Path)diff --git a/tools/vqgan/inference.py b/tools/vqgan/inference.pyindex 17c9034..000b5a4 100644--- a/tools/vqgan/inference.py+++ b/tools/vqgan/inference.py@@ -59,7 +59,7 @@ def load_model(config_name, checkpoint_path, device="cuda"):@click.option("--config-name", default="firefly_gan_vq")@click.option("--checkpoint-path",- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",)@click.option("--device",diff --git a/tools/webui.py b/tools/webui.pyindex da19f6b..0aef2d6 100644--- a/tools/webui.py+++ b/tools/webui.py@@ -560,12 +560,12 @@ def parse_args():parser.add_argument("--llama-checkpoint-path",type=Path,- default="checkpoints/fish-speech-1.2-sft",+ default="checkpoints/fish-speech-1.4",)parser.add_argument("--decoder-checkpoint-path",type=Path,- default="checkpoints/fish-speech-1.2-sft/firefly-gan-vq-fsq-4x1024-42hz-generator.pth",+ default="checkpoints/fish-speech-1.4/firefly-gan-vq-fsq-8x1024-21hz-generator.pth",)parser.add_argument("--decoder-config-name", type=str, default="firefly_gan_vq")parser.add_argument("--device", type=str, default="cuda")