LLM inference in C/C++
C++ ggml
model : drop the redundant precision overrides in the Kolibri-1 router (#30865)
Georgi Gerganov9df0b73c7079
diff --git a/src/models/kolibri1.cpp b/src/models/kolibri1.cppindex acc1dd1592..2301ef272a 100644--- a/src/models/kolibri1.cpp+++ b/src/models/kolibri1.cpp@@ -131,11 +131,8 @@ llama_model_kolibri1::graph::graph(const llama_model & model, const llm_graph_pacur = build_norm(ffn_inp, layer.ffn_norm, NULL, LLM_NORM_RMS, il);cb(cur, "ffn_norm", il);- // The router runs in f32: the selection adds the bias to the logits, the weights- // are the sigmoid of the unbiased logits, so the same logits feed both.+ // The selection adds the bias to the logits, the weights are the sigmoid of the unbiased logits, so the same logits feed both.ggml_tensor * logits = build_lora_mm(layer.ffn_gate_inp, cur);- ggml_prec_set_acc(logits, GGML_PREC_F32);- ggml_prec_set_src(logits, GGML_PREC_F32, 1);cb(logits, "ffn_moe_logits", il);ggml_tensor * selection = ggml_add(ctx0, logits, layer.ffn_exp_probs_b);