37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
C avx2ccpu-inferenceggufinference-enginelinuxllmlocal-llmmixture-of-expertsmoenvmequantizationqwen3simdspeculative-decodingvulkan
Commits on main
Release 2026-10-13 (6): a build without GPU support links again, and a glslang too old for the shaders gives one, saying why (issue #22); issue #21 was #17's
94b72c9 camauri committed 1 month ago
Release 2026-10-13 (5): vkcheck in the binary packages for Linux and Windows
4df4970 camauri committed 1 month ago
Release 2026-10-13 (4): vkcheck tells whether a GPU can run Janas-LLM's GPU backend, macOS with MoltenVK included (issue #20)
19c8084 camauri committed 1 month ago
Release 2026-10-13 (3): Qwen3-Coder-Next as llama.cpp converts it now, its MXFP4 shared experts written as Q8_0 exactly (issue #18)
b51ed19 camauri committed 1 month ago
Release 2026-10-13 (2): a discrete GPU's results reach the CPU again (issue #17), janas-bench says that the GPU reads a prompt in every configuration (issue #16)
605b286 camauri committed 1 month ago
Release 2026-10-13: the GPU's products of 2 to 8 vectors faster (Q4_K, Q6_K, Q5_K, IQ4_XS), Qwen3.8-27B reading prompts whole on the GPU (30.6 tok/s), a qwen3.5 model's drafts verifiable on the GPU (JANAS_GPU_SHORT=1)
092e33a camauri committed 1 month ago
Release 2026-10-12 (4): janas-bench takes a model by name, from the list, or a file
98c8300 camauri committed 1 month ago
Release 2026-10-12 (3): the cores' efficiency class read with older MinGW-w64 headers too
06ec016 camauri committed 1 month ago
Release 2026-10-12 (2): the Windows build's packages installed with no question asked
69f47d9 camauri committed 1 month ago
Release 2026-10-12: Janas on Windows - the same sources built for Windows 10 and 11 x86-64, as fast as on Linux on the development laptop
0b0142b camauri committed 1 month ago
Release 2026-10-11 (8): the README split into pages under docs/, and the wiki written from them at each release
cafa34b camauri committed 1 month ago
Release 2026-10-11 (7): Q3_K products of a prompt on the GPU; the engine named Janas-LLM throughout
60a56d6 camauri committed 1 month ago
Release 2026-10-11 (6): the measures of 11 October - Qwen3-Coder-Next 66 against 14 tok/s reading, Qwen3.6-35B-A3B 104 against 79 - and the test machine's operating system
3d64816 camauri committed 1 month ago
Release 2026-10-11 (5): the README opens with the speeds against llama.cpp and the binary package
4644ca0 camauri committed 1 month ago
Release 2026-10-11 (2): the binary package builds on Ubuntu 22.04
8fcfc85 camauri committed 1 month ago
Release 2026-10-11: a mixture's prompt layer by layer on the GPU, its experts read ahead (Qwen3.6-35B-A3B +19%); models downloaded already converted from Hugging Face; a binary package built by GitHub Actions; the tuner leaves cores another program has taken
ab26273 camauri committed 1 month ago
Release 2026-10-10 (3): Qwen3.6-35B-A3B and Qwen3-Next-80B-A3B against llama.cpp in the README's comparison
1323661 camauri committed 1 month ago
Release 2026-10-10 (2): the documentation brought up to date - the tuner's four kinds of pass, the prompt's blocks of 512, the troubleshooting variables
ce9e06b camauri committed 1 month ago
Release 2026-10-10: prompts read on the GPU, faster than llama.cpp on the same GPU; Qwen3.5's recurrent layers on the GPU; the fast prompt on by default
da0a994 camauri committed 1 month ago
Release 2026-10-09 (3): Qwen3-4B and Qwen3-30B-A3B also with an importance matrix; fast profiles for the new versions
4d2d719 camauri committed 1 month ago
Release 2026-10-09 (2): the Q3_K product a quarter faster, the three-bit Qwen3-30B-A3B as fast as the four-bit one held in memory
945eec6 camauri committed 1 month ago
Release 2026-10-09: Qwen3-30B-A3B in unsloth's three-bit mix, smaller and closer to the original
4b73e1d camauri committed 1 month ago
Release 2026-10-08 (4): light experts not in memory left out instead of fewer experts per token; the machine's measure without a head start
1a8d6f7 camauri committed 1 month ago
Release 2026-10-08 (3): the catalog says what each small model is for - Qwen3.5-0.8B a test model, Qwen3.5-2B the small one for tools
2891bd1 camauri committed 1 month ago
Release 2026-10-08 (2): the weather of a region, cities by their Italian names, each model's own layout translations
bfb1d26 camauri committed 1 month ago
Release 2026-10-08: models by name with measured fast profiles and a measure on the machine; the output head at four bits on request; faster kernels and attention
b04ce09 camauri committed 1 month ago
Release 2026-10-07: small gains kept - prompts in blocks, drafts checked, long contexts; the README on the kernels
fe33e68 camauri committed 1 month ago