37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
C avx2ccpu-inferenceggufinference-enginelinuxllmlocal-llmmixture-of-expertsmoenvmequantizationqwen3simdspeculative-decodingvulkan
{
"name": "janas-chat-demo",
"private": true,
"version": "0.1.0",
"description": "A chat in TypeScript on Janas-LLM inside Node.js: the model in the server's process, its replies streamed to the browser",
"license": "GPL-3.0-or-later",
"type": "module",
"engines": {
"node": ">=20"
},
"scripts": {
"build": "tsc -p tsconfig.server.json && tsc -p tsconfig.web.json",
"check": "tsc -p tsconfig.server.json --noEmit && tsc -p tsconfig.web.json --noEmit",
"start": "node dist/node/server/main.js"
},
"devDependencies": {
"@types/node": "^20.19.43",
"typescript": "^7.0.2"
}
}