Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
Remove n-gpu-layer limitation (#534)
BIGWONG01b8d49443ea
diff --git a/llama.cpp/llama.cpp b/llama.cpp/llama.cppindex 18da7731..8187c5ae 100644--- a/llama.cpp/llama.cpp+++ b/llama.cpp/llama.cpp@@ -5682,7 +5682,7 @@ static bool llm_load_tensors(auto & hparams = model.hparams;- n_gpu_layers = std::min(n_gpu_layers, (int)hparams.n_layer); // [jart]+ // n_gpu_layers = std::min(n_gpu_layers, (int)hparams.n_layer); // [jart]n_gpu_layers = std::max(n_gpu_layers, 0); // [jart]model.split_mode = split_mode;