Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
Dev gzf (#1475)
zhifu gao68f0603b10d1
diff --git a/.github/ISSUE_TEMPLATE/ask_questions.md b/.github/ISSUE_TEMPLATE/ask_questions.mdindex 23a5a7f0b..4716faa9a 100644--- a/.github/ISSUE_TEMPLATE/ask_questions.md+++ b/.github/ISSUE_TEMPLATE/ask_questions.md@@ -4,8 +4,12 @@ about: If you have questions, please first search existing issues and docslabels: 'question, needs triage'---+Notice: In order to resolve issues more efficiently, please raise issue following the template.+(注意:为了更加高效率解决您遇到的问题,请按照模板提问,补充细节)+## ❓ Questions and Help+### Before asking:1. search the issues.2. search the docs.diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.mdindex 82ba025fc..80b03179a 100644--- a/.github/ISSUE_TEMPLATE/bug_report.md+++ b/.github/ISSUE_TEMPLATE/bug_report.md@@ -4,6 +4,9 @@ about: Submit a bug report to help us improvelabels: 'bug, needs triage'---+Notice: In order to resolve issues more efficiently, please raise issue following the template.+(注意:为了更加高效率解决您遇到的问题,请按照模板提问,补充细节)+## 🐛 Bug<!-- A clear and concise description of what the bug is. -->diff --git a/examples/industrial_data_pretraining/llm_asr/conf/whisper_qwen_linear.yaml b/examples/industrial_data_pretraining/llm_asr/conf/whisper_qwen_linear.yamlindex 4603f805b..fb00b122c 100644--- a/examples/industrial_data_pretraining/llm_asr/conf/whisper_qwen_linear.yaml+++ b/examples/industrial_data_pretraining/llm_asr/conf/whisper_qwen_linear.yaml@@ -12,17 +12,16 @@ model_conf:length_normalized_loss: true# encoder-audio_encoder: iic/Whisper-large-v2 #iic/Whisper-large-v3+audio_encoder: "/nfs/zhifu.gzf/init_model/Whisper-large-v3" #iic/Whisper-large-v3audio_encoder_conf:hub: msfreeze: true- init_param_path: "/nfs/maziyang.mzy/models/vicuna-7b-v1.5"-llm: Vicuna+llm: Qwen1.5-7b-chatllm_conf:hub: hffreeze: true- init_param_path: "/nfs/maziyang.mzy/models/vicuna-7b-v1.5"+ init_param_path: "/nfs/zhifu.gzf/init_model/qwen/Qwen1___5-7B-Chat"audio_adaptor: Linearaudio_adaptor_conf:@@ -34,7 +33,7 @@ audio_adaptor_conf:frontend: WhisperFrontendfrontend_conf:fs: 16000- whisper_model: large-v2+ whisper_model: large-v3do_pad_trim: truepermute: true # true: [bs, frames, dims]; false: [bs, dims, frames]@@ -66,8 +65,9 @@ train_conf:optim: adamwoptim_conf:lr: 0.0001- weight_decay: 0.000001-scheduler: warmuplr+ weight_decay: 0.000000++scheduler: custom_lambdalrscheduler_conf:warmup_steps: 1000@@ -83,12 +83,12 @@ dataset_conf:preprocessor_text: TextPreprocessRemovePunctuationaudio_adaptor_downsample_rate: ${audio_adaptor_conf.downsample_rate}audio_encoder_downsample_rate: 2- prompt: "<|startoftranscription|><|zh|><|transcribe|><|zh|><|notimestamps|><|wo_itn|>"+# prompt: "<|startoftranscription|><|zh|><|transcribe|><|zh|><|notimestamps|><|wo_itn|>"tokenizer: HuggingfaceTokenizertokenizer_conf:unk_symbol: <unk>- init_param_path: "/nfs/maziyang.mzy/models/vicuna-7b-v1.5"+ init_param_path: "/nfs/zhifu.gzf/init_model/qwen/Qwen1___5-7B-Chat"diff --git a/funasr/datasets/audio_datasets/index_ds.py b/funasr/datasets/audio_datasets/index_ds.pyindex 12ffd23d5..34f7b4fa6 100644--- a/funasr/datasets/audio_datasets/index_ds.py+++ b/funasr/datasets/audio_datasets/index_ds.py@@ -99,7 +99,8 @@ class IndexDSJsonlRankFull(torch.utils.data.Dataset):target = data["target"]source_len = data.get("source_len", 1)target_len = data.get("target_len", 0)-+ if "aishell" in source:+ target = target.replace(" ", "")contents.append({"source": source,"prompt": prompt,"target": target,