FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
Commits on main
feat(server)!: add --gpu to choose the GPU on multi-GPU machines (#117)
2757bb5 Xiaoze Fan committed 2 months ago
fix(checkpoint): FTW truncated shard raises OSError instead of silent garbage load (#25)
bd372b6 Lancer committed 2 months ago
Merge pull request #112 from FlashML-org/feat/split-residency
e0a3bbc Shuo Yang committed 2 months ago
feat(moe): estimate expert-bank bytes from the model config
eebb3f5 Xiaoze Fan committed 2 months ago
feat(moe): per-layer host-bank residency -- split lock-CPU/pin-GPU layers, auto selection under capped pin quota
c41833b Xiaoze Fan committed 2 months ago
build(kernel-cache): add sm_80 (A100/A800) to default arches (#75)
184a4f1 jasonfan committed 2 months ago
docs(README): use absolute URLs so PyPI renders assets and links
3f02e3c jason-fxz committed 3 months ago
Merge branch 'dev/refine-readme': README overhaul + quick start guide
c0a4a60 jason-fxz committed 3 months ago
build(release): pypi-ready wheels, metadata, and publish workflow (#6)
b86a904 Xiaoze Fan committed 3 months ago
build(release): pypi-ready wheels, metadata, and publish workflow
76edc16 Xiaoze Fan committed 3 months ago