FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
Merge pull request #112 from FlashML-org/feat/split-residency
Shuo Yange0a3bbc04652
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.