Skip to content

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

Commits on main

07d143f
Implement custom github release action
Daniel Hiltgen committed 2 years ago
a12283e
Merge pull request #5037 from dhiltgen/faster_win_build
Daniel Hiltgen committed 2 years ago
4b0050c
More parallelism on windows generate
Daniel Hiltgen committed 2 years ago
0577af9
17ce203
Merge pull request #4874 from dhiltgen/rocm_v6_bump
Daniel Hiltgen committed 2 years ago
d76555f
2786dff
gpu: Fix build warning
Lei Jitang committed 2 years ago
225f0d1
Merge pull request #4972 from jayson-cloude/main
Daniel Hiltgen committed 2 years ago
532db58
Centralize GPU configuration vars
Daniel Hiltgen committed 2 years ago
6be309e
Workaround gfx900 SDMA bugs
Daniel Hiltgen committed 2 years ago
da3bf23
Bump ROCm linux to 6.1.1
Daniel Hiltgen committed 2 years ago
26ab677
Merge pull request #4517 from dhiltgen/gpu_incremental
Daniel Hiltgen committed 2 years ago
45cacba
Remove mmap related output calc logic
Daniel Hiltgen committed 2 years ago
17df652
review comments and coverage
Daniel Hiltgen committed 2 years ago
6f351bf
Prevent multiple concurrent loads on the same gpus
Daniel Hiltgen committed 2 years ago
ff4f0cb
Refine CPU load behavior with system memory visibility
Daniel Hiltgen committed 2 years ago
fc37c19
Reintroduce nvidia nvml library for windows
Daniel Hiltgen committed 2 years ago
434dfe3
Refactor intel gpu discovery
Daniel Hiltgen committed 2 years ago
4e2b7e1
Harden unload for empty runners
Daniel Hiltgen committed 2 years ago
48702dd
refined test timing
Daniel Hiltgen committed 2 years ago
68dfc62
Support forced spreading for multi GPU
Daniel Hiltgen committed 2 years ago
5e8ff55
Improve multi-gpu handling at the limit
Daniel Hiltgen committed 2 years ago
6fd04ca
Fix concurrency integration test to work locally
Daniel Hiltgen committed 2 years ago
206797b
Refine GPU discovery to bootstrap once
Daniel Hiltgen committed 2 years ago
43ed358
Use DRM driver for VRAM info for amd
Daniel Hiltgen committed 2 years ago
b32ebb4
Fix server.cpp for the new cuda build macros
Daniel Hiltgen committed 2 years ago
fb9cdfa
Revert "Limit GPU lib search for now (#4777)"
Daniel Hiltgen committed 2 years ago
efac488
6b800aa