LLM inference in C/C++
C++ ggml
memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967)
fairydreamingbdf3955159d7
diff --git a/src/llama-kv-cache.cpp b/src/llama-kv-cache.cppindex 8fafcd1530..65afbd8c37 100644--- a/src/llama-kv-cache.cpp+++ b/src/llama-kv-cache.cpp@@ -2034,7 +2034,7 @@ void llm_graph_input_k_shift::set_input(const llama_ubatch * ubatch) {kv_self->set_input_k_shift(k_shift);}- if (k_rot) {+ if (k_rot && k_rot->buffer) {kv_self->set_input_k_rot(k_rot);}}