Distribute and run LLMs with a single file.
C++ cross-platformggufllama-cpplocal-ailocal-inferencelocal-llmopen-source-aisingle-file-executablespeech-to-text
Clamp out of range values in K quantizer
Justine Tunneyef0307ea62c8
diff --git a/llama.cpp/ggml-quants.c b/llama.cpp/ggml-quants.cindex e2ec5e79..bde8c846 100644--- a/llama.cpp/ggml-quants.c+++ b/llama.cpp/ggml-quants.c@@ -1314,7 +1314,11 @@ void dequantize_row_q8_0(const block_q8_0 * restrict x, float * restrict y, int// ===================== Helper functions//static inline int nearest_int(float fval) {- assert(fval <= 4194303.f);++ // [jart] https://github.com/ggerganov/llama.cpp/issues/2982+ // assert(fval <= 4194303.f);+ fval = fminf(fval, 4194303.f);+float val = fval + 12582912.f;int i; memcpy(&i, &val, sizeof(int));return (i & 0x007fffff) - 0x00400000;