Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Commits on main
Use slot with cached prompt instead of least recently used (#5492)
d89454d Jeffrey Morgan committed 2 years ago
Merge pull request #5469 from dhiltgen/prevent_system_oom
af28b94 Daniel Hiltgen committed 2 years ago
Merge pull request #5466 from dhiltgen/fix_clip_unicode
02c24d3 Daniel Hiltgen committed 2 years ago
fix error detection by limiting model loading error parsing (#5472)
4d71c55 Jeffrey Morgan committed 2 years ago
Merge pull request #5243 from dhiltgen/modelfile_use_mmap
ccd7785 Daniel Hiltgen committed 2 years ago
Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)
3b5a4a7 royjhan committed 2 years ago
Merge pull request #5467 from dhiltgen/bogus_cpu_mac_error
daed063 Daniel Hiltgen committed 2 years ago
Merge pull request #5465 from dhiltgen/better_cuda_logging
0d4dd70 Daniel Hiltgen committed 2 years ago
Merge pull request #5442 from dhiltgen/concurrency_docs
d2f1902 Daniel Hiltgen committed 2 years ago
OpenAI: /v1/models and /v1/models/{model} compatibility (#5007)
996bb1b royjhan committed 2 years ago