Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
vad
游雁1b21c1120c44
diff --git a/README.md b/README.mdindex 454adc956..22c53da4f 100644--- a/README.md+++ b/README.md@@ -153,6 +153,8 @@ wav_file = f"{model.model_path}/example/asr_example.wav"res = model.generate(input=wav_file)print(res)```+Note: The output format of the VAD model is: `[[beg1, end1], [beg2, end2], ..., [begN, endN]]`, where `begN/endN` indicates the starting/ending point of the `N-th` valid audio segment, measured in milliseconds.+### Voice Activity Detection (Streaming)```pythonfrom funasr import AutoModel@@ -175,6 +177,13 @@ for i in range(total_chunk_num):if len(res[0]["value"]):print(res)```+Note: The output format for the streaming VAD model can be one of four scenarios:+- `[[beg1, end1], [beg2, end2], .., [begN, endN]]`:The same as the offline VAD output result mentioned above.+- `[[beg, -1]]`:Indicates that only a starting point has been detected.+- `[[-1, end]]`:Indicates that only an ending point has been detected.+- `[]`:Indicates that neither a starting point nor an ending point has been detected.++The output is measured in milliseconds and represents the absolute time from the starting point.### Punctuation Restoration```pythonfrom funasr import AutoModeldiff --git a/README_zh.md b/README_zh.mdindex 07cdd1f17..63ad2e25e 100644--- a/README_zh.md+++ b/README_zh.md@@ -101,10 +101,8 @@ funasr ++model=paraformer-zh ++vad_model="fsmn-vad" ++punc_model="ct-punc" ++inpfrom funasr import AutoModel# paraformer-zh is a multi-functional asr model# use vad, punc, spk or not as you need-model = AutoModel(model="paraformer-zh", model_revision="v2.0.4",- vad_model="fsmn-vad", vad_model_revision="v2.0.4",- punc_model="ct-punc-c", punc_model_revision="v2.0.4",- # spk_model="cam++", spk_model_revision="v2.0.2",+model = AutoModel(model="paraformer-zh", vad_model="fsmn-vad", punc_model="ct-punc-c",+ # spk_model="cam++")res = model.generate(input=f"{model.model_path}/example/asr_example.wav",batch_size_s=300,@@ -122,7 +120,7 @@ chunk_size = [0, 10, 5] #[0, 10, 5] 600ms, [0, 8, 4] 480msencoder_chunk_look_back = 4 #number of chunks to lookback for encoder self-attentiondecoder_chunk_look_back = 1 #number of encoder chunks to lookback for decoder cross-attention-model = AutoModel(model="paraformer-zh-streaming", model_revision="v2.0.4")+model = AutoModel(model="paraformer-zh-streaming")import soundfileimport os@@ -146,19 +144,21 @@ for i in range(total_chunk_num):```pythonfrom funasr import AutoModel-model = AutoModel(model="fsmn-vad", model_revision="v2.0.4")+model = AutoModel(model="fsmn-vad")wav_file = f"{model.model_path}/example/asr_example.wav"res = model.generate(input=wav_file)print(res)```+注:VAD模型输出格式为:`[[beg1, end1], [beg2, end2], .., [begN, endN]]`,其中`begN/endN`表示第`N`个有效音频片段的起始点/结束点,+单位为毫秒。### 语音端点检测(实时)```pythonfrom funasr import AutoModelchunk_size = 200 # ms-model = AutoModel(model="fsmn-vad", model_revision="v2.0.4")+model = AutoModel(model="fsmn-vad")import soundfile@@ -175,12 +175,18 @@ for i in range(total_chunk_num):if len(res[0]["value"]):print(res)```+注:流式VAD模型输出格式为4种情况:+- `[[beg1, end1], [beg2, end2], .., [begN, endN]]`:同上离线VAD输出结果。+- `[[beg, -1]]`:表示只检测到起始点。+- `[[-1, end]]`:表示只检测到结束点。+- `[]`:表示既没有检测到起始点,也没有检测到结束点+输出结果单位为毫秒,从起始点开始的绝对时间。### 标点恢复```pythonfrom funasr import AutoModel-model = AutoModel(model="ct-punc", model_revision="v2.0.4")+model = AutoModel(model="ct-punc")res = model.generate(input="那今天的会就到这里吧 happy new year 明年见")print(res)@@ -190,7 +196,7 @@ print(res)```pythonfrom funasr import AutoModel-model = AutoModel(model="fa-zh", model_revision="v2.0.0")+model = AutoModel(model="fa-zh")wav_file = f"{model.model_path}/example/asr_example.wav"text_file = f"{model.model_path}/example/text.txt"diff --git a/examples/industrial_data_pretraining/fsmn_vad_streaming/demo.py b/examples/industrial_data_pretraining/fsmn_vad_streaming/demo.pyindex f04312342..c28db7a07 100644--- a/examples/industrial_data_pretraining/fsmn_vad_streaming/demo.py+++ b/examples/industrial_data_pretraining/fsmn_vad_streaming/demo.py@@ -10,6 +10,8 @@ model = AutoModel(model="damo/speech_fsmn_vad_zh-cn-16k-common-pytorch", model_rres = model.generate(input=wav_file)print(res)+# [[beg1, end1], [beg2, end2], .., [begN, endN]]+# beg/end: ms@@ -37,3 +39,9 @@ for i in range(total_chunk_num):# print(res)if len(res[0]["value"]):print(res)+++# 1. [[beg1, end1], [beg2, end2], .., [begN, endN]]; [[beg, end]]; [[beg1, end1], [beg2, end2]]+# 2. [[beg, -1]]+# 3. [[-1, end]]+# beg/end: ms\ No newline at end of file