Skip to content

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

Commits on main

6eaf194
llm: New memory management
Jesse Gross committed 1 year ago
d5a0d8d
ef7d26b
update vendored llama.cpp and ggml (#11823)
Michael Yang committed 1 year ago
1a19df1
7ccfd97
test: add valid responses (#11902)
Daniel Hiltgen committed 1 year ago
c385ca8
discovery: fix cudart driver version (#11614)
Daniel Hiltgen committed 1 year ago
837379a
a24f906
cuda: leverage JIT for smaller footprint (#11635)
Daniel Hiltgen committed 1 year ago
dc5a645
bb71654
fix(openai): handle reasoning_effort (#11868)
Michael Yang committed 1 year ago
d0cf6c8
discover: CPU supports flash attention
Jesse Gross committed 1 year ago
8f4ec9a
ee04dbb
sched: Add support for grouping GPUs (#10678)
Daniel Andersen committed 1 year ago
ea7657b
2c776f0
ggml: No-alloc mode
Jesse Gross committed 1 year ago
79f6376
ggml: Support closing backends
Jesse Gross committed 1 year ago
756c78c
ggml: Use GGML's typedef'ed pointer types
Jesse Gross committed 1 year ago
d7f4f78
tests: add integration coverage for oss-gpt (#11696)
Daniel Hiltgen committed 1 year ago
114c3f2
f2e9c9a
openai: always provide reasoning
Devon Rifkin committed 1 year ago
735c41f
docs: update the faq (#11760)
Patrick Devine committed 1 year ago
44bc36d
clean up debugging (#11756)
Daniel Hiltgen committed 1 year ago
fa8be9e