Skip to content

LLM inference in C/C++

C++ ggml

memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967)

fairydreamingbdf3955159d7
diff --git a/src/llama-kv-cache.cpp b/src/llama-kv-cache.cpp
index 8fafcd1530..65afbd8c37 100644
--- a/src/llama-kv-cache.cpp
+++ b/src/llama-kv-cache.cpp
@@ -2034,7 +2034,7 @@ void llm_graph_input_k_shift::set_input(const llama_ubatch * ubatch) {
kv_self->set_input_k_shift(k_shift);
}
- if (k_rot) {
+ if (k_rot && k_rot->buffer) {
kv_self->set_input_k_rot(k_rot);
}
}