LLM inference in C/C++
C++ ggml
llama.cpp/examples/speculative
Demonstration of speculative decoding and tree-based speculative decoding techniques
More info:
LLM inference in C/C++
Demonstration of speculative decoding and tree-based speculative decoding techniques
More info: