FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
Commits on main
feat(gemma4): serve image input on the Gemma-4 releases (#467)
84d236c Xiaoze Fan committed 2 months ago
fix(models): compute shared experts before in-place routed experts (#463)
e0886cc cherry77-cloud committed 2 months ago
feat(server): carry tool-result images on the following user turn
db64879 Xiaoze Fan committed 2 months ago
feat(launch): declare image input to the agents when the server serves it
8ff0cce Xiaoze Fan committed 2 months ago
fix(quant): load ModelOpt exports that ship no input_scale (#462)
f7dbab7 Xiaoze Fan committed 2 months ago
fix(server): honor configured output default across APIs (#411)
9535656 Earl Co committed 2 months ago
feat(qwen3_5_moe): read every checkpoint layout through the QuantConfig (#438)
0ffd5c8 Xiaoze Fan committed 2 months ago
fix(scheduler): reserve paged KV at allocation granularity (#367)
46d2743 cherry77-cloud committed 2 months ago
feat(qwen4_exp): load block-fp8 dense projections natively (#428)
ddd2e3a Xiaoze Fan committed 2 months ago
refactor(quant): hand the checkpoint QuantConfig to the weight readers (#427)
fb7f732 Xiaoze Fan committed 2 months ago
chore(scripts): add ftw_hotfix.py to repair FTW checkpoints from older builds (#424)
505477a Xiaoze Fan committed 2 months ago
fix(qwen4_exp): take the expert quant kind from the checkpoint QuantConfig (#426)
fa814ab Xiaoze Fan committed 2 months ago
fix(checkpoint): write the Qwen3.8-Flash-Next PLE table next to the FTW (#420)
3d919e9 Xiaoze Fan committed 2 months ago
ci: publish nightly wheels to a rolling nightly release on this repo
7dfc37a Xiaoze Fan committed 2 months ago
refactor(quant): config, scheme and method layers for quantization (#418)
477c860 Xiaoze Fan committed 2 months ago
ci: publish engine-<platform>.json manifests alongside the nightly wheels (#377)
af71ba4 Xiaoze Fan committed 2 months ago
chore: add issue templates with FAQ/Roadmap checks and auto-labels (#375)
86214a9 Xiaoze Fan committed 2 months ago
fix(kernel): make Triton top-k/top-p sampling exact (#329)
03c28d2 cherry77-cloud committed 2 months ago
fix(models): detect nvfp4 experts behind a mixed-precision compressed-tensors format (#343)
6eca2d7 Xiaoze Fan committed 2 months ago
fix(hf): download the shards the safetensors index names (#336)
a80b4d3 Xiaoze Fan committed 2 months ago
feat(qwen4_exp): stream the PLE n-gram table from disk (#311)
4c0bad3 Xiaoze Fan committed 2 months ago
perf(moe): route fused_topk through the in-repo triton router (#319)
e05cff8 Xiaoze Fan committed 2 months ago
fix(kernel): unbreak the nightly kernel-cache wheel build (#310)
3a20a79 Xiaoze Fan committed 2 months ago
fix(kernel): avoid row-wise _scaled_mm stall on sm_89 with torch<2.12 (#243)
58f4b9e Berni McCoy committed 2 months ago