FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
Commits on main
feat(server): enable reasoning_effort on /v1/chat/completions (#2)
4f510f1 Xiaoze Fan committed 3 months ago
build(deps): pin apache-tvm-ffi, flashlib, triton to exact versions
3e5214d jasonfan committed 3 months ago
feat(kvcache): checkpoint decode state at the tool-call opener token
ea5348b Xiaoze Fan committed 3 months ago
fix(gemma4): read head_dim / kv heads per attention type from per_layer_config
ee54f63 Xiaoze Fan committed 3 months ago