Skip to content

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

mlxrunner: release the buffers weight loading leaves in the MLX pool

Jesse Grossb68b112bd886
diff --git a/x/mlxrunner/runner.go b/x/mlxrunner/runner.go
index 626353c9c..4814edfd5 100644
--- a/x/mlxrunner/runner.go
+++ b/x/mlxrunner/runner.go
@@ -63,6 +63,7 @@ func (r *Runner) Load(modelName string) error {
return err
}
mlx.Eval(weights...)
+ mlx.ClearCache()
r.weights = mlx.NewScope()
r.weights.Attach(weights...)
configureWiredMemory()