A high-performance inference engine for AI models
Commits on main
[feat] Replace NGramSpeculator with tagged multi-table backoff + KN (#270)
e9f08c7 hikettei committed 8 months ago
Use QuantizationMode as specializable parameter (#267)
ede4975 Aleksandr Golokoz committed 8 months ago