Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Commits on main
server: align generate with native chat templates (#16878)
16739de Daniel Hiltgen committed 4 months ago
llm: fix ollama ps double-counting mmap'd weights on partial offload (#16709)
0463940 Philip Sinitsin committed 4 months ago
llama: default qwen2.5vl window attention metadata (#16868)
33878e6 Daniel Hiltgen committed 4 months ago
llm: preserve generation headroom for shifted prompts (#16856)
c191a14 Parth Sareen committed 5 months ago
llama: add sm_86 architecture to cuda_v13_windows preset (#16834)
46bc1bc anish committed 5 months ago
launch/codex: detect model drift when Codex App UI switches away from Ollama (#16864)
2a8b315 Bruce MacDonald committed 5 months ago
mlxrunner: choose the speculative draft length to maximize throughput
505e35f Jesse Gross committed 5 months ago
mlxrunner: resolve each speculative round in one host sync
1148751 Jesse Gross committed 5 months ago
mlxrunner: apply in-flight drafts to proposal penalty history
f93efe2 Jesse Gross committed 5 months ago
mlxrunner: support draft heads that maintain draft caches
28fbbb0 Jesse Gross committed 5 months ago
mlxrunner: host speculative decoding in the text generation pipeline
340c51b Jesse Gross committed 5 months ago
discover: fix inverted iGPU/dGPU Vulkan classification on Windows hybrid graphics (#16669)
fc58544 Sahil Kadadekar committed 5 months ago
models: add cohere2_moe (Command A / North) to the MLX engine (#16670)
acfb50d Jeffrey Morgan committed 5 months ago
integration: look for the "hf" tool in integration tests (#16765)
9e4ed74 Patrick Devine committed 5 months ago
server: context shift for context windows larger than 8k, add error when hitting context limit (#16712)
bbb40a0 Jeffrey Morgan committed 5 months ago