Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
Sanitize -ngl flag on Apple Metal
Justine Tunney79b88f8ddcc2
diff --git a/llamafile/gpu.c b/llamafile/gpu.cindex 2a1bfb5e..440a5b8e 100644--- a/llamafile/gpu.c+++ b/llamafile/gpu.c@@ -70,7 +70,7 @@ int llamafile_gpu_layers(int n_gpu_layers) {}// Apple Metal is safe enough to enable by default.- if (n_gpu_layers == -1 && ggml_metal_supported()) {+ if ((n_gpu_layers < 0 || n_gpu_layers > 1) && ggml_metal_supported()) {n_gpu_layers = 1;}