Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
fix(vulkan): refresh ggml for AMD submission handling (#3482)
zhifu gao500956bc331b
diff --git a/runtime/llama.cpp/CMakeLists.txt b/runtime/llama.cpp/CMakeLists.txtindex 66fa28fd9..5a6948ac1 100644--- a/runtime/llama.cpp/CMakeLists.txt+++ b/runtime/llama.cpp/CMakeLists.txt@@ -34,7 +34,9 @@ set(LLAMA_BUILD_SERVER OFF CACHE BOOL "" FORCE)set(LLAMA_CURL OFF CACHE BOOL "" FORCE)FetchContent_Declare(llamaGIT_REPOSITORY https://github.com/ggml-org/llama.cpp.git- GIT_TAG 8086439a4cea94c71a5dfb8fe4ad1546aebd640f)+ # Includes the Vulkan submission-batching and DeviceLost diagnostics fixes+ # needed by recent AMD drivers (ggml-org/llama.cpp#26371).+ GIT_TAG 803b7fcae893e9caaee3921779628fef83ac0965)FetchContent_GetProperties(llama)if(NOT llama_POPULATED)FetchContent_Populate(llama)diff --git a/runtime/llama.cpp/README.md b/runtime/llama.cpp/README.mdindex 93194553b..5d5ee2c9a 100644--- a/runtime/llama.cpp/README.md+++ b/runtime/llama.cpp/README.md@@ -187,6 +187,28 @@ cmake --build build-vulkan --config Release --target llama-funasr-sensevoice`--backend cpu` remains the default. The Windows Vulkan package currentlyaccelerates SenseVoiceSmall only, matching the Linux Vulkan package.+#### AMD Windows troubleshooting++Recent AMD drivers can report `VK_ERROR_DEVICE_LOST` or terminate the process+when a graph submission exceeds the driver's timeout. The pinned ggml revision+reduces submission sizes on smaller AMD GPUs and fixes the batching threshold.+To force smaller submissions or collect the last submitted tensors for a bug+report, run from PowerShell with:++```powershell+$env:GGML_VK_MAX_NODES_PER_SUBMIT = "16"+$env:GGML_VK_SERIALIZE_SUBMISSIONS = "1"+.\llama-funasr-sensevoice.exe `+ -m sensevoice-small-f16.gguf --vad fsmn-vad.gguf -a sample.wav --backend vulkan+```++Try `8` or `1` if `16` still triggers a driver timeout. Include the complete+stderr output, GPU model, and driver version when reporting a failure. Remove+the variables after diagnosis because serial submissions can reduce throughput.+Use `--backend cpu` as the reliable fallback on affected driver/device pairs.+The current SenseVoiceSmall graph does not create a flash-attention operation,+so a `--no-flash-attn` switch would not change this execution path.+## Build (shared)```bashgit clone https://github.com/ggml-org/llama.cpp && cd llama.cppdiff --git a/runtime/llama.cpp/fun-asr-nano/funasr-cli/funasr-cli.cpp b/runtime/llama.cpp/fun-asr-nano/funasr-cli/funasr-cli.cppindex eb3b6f0b2..7d0929f5f 100644--- a/runtime/llama.cpp/fun-asr-nano/funasr-cli/funasr-cli.cpp+++ b/runtime/llama.cpp/fun-asr-nano/funasr-cli/funasr-cli.cpp@@ -208,7 +208,7 @@ int main(int argc,char**argv){llama_context*ctx=llama_init_from_model(model,cp);if(!ctx){fprintf(stderr,"failed to create llama context\n");llama_model_free(model);return 1;}auto sp=llama_sampler_chain_default_params(); llama_sampler*smpl=llama_sampler_chain_init(sp);- if(rep!=1.0f) llama_sampler_chain_add(smpl,llama_sampler_init_penalties(256,rep,0.0f,0.0f));+ if(rep!=1.0f) llama_sampler_chain_add(smpl,llama_sampler_init_penalties(llama_vocab_n_tokens(vocab),256,rep,0.0f,0.0f));llama_sampler_chain_add(smpl,llama_sampler_init_greedy());const char*prefix="<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\n语音转写:";