37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
C avx2ccpu-inferenceggufinference-enginelinuxllmlocal-llmmixture-of-expertsmoenvmequantizationqwen3simdspeculative-decodingvulkan
/* SPDX-License-Identifier: GPL-3.0-or-later
Copyright (C) 2026 Maurizio Cammalleri */
/*
* sysinfo_os.h - what each system tells of its CPUs, for sysinfo.c:
* Linux's in sysinfo.c, Windows's in sysinfo_win.c.
*/
#ifndef JANAS_COMMON_SYSINFO_OS_H
#define JANAS_COMMON_SYSINFO_OS_H
/* The CPUs the calling thread may use, at most max. The count, or -1. */
int janas_os_allowed_cpus(int *out, int max);
/* For each of the nc CPUs of cand: its kind (0 performance, 1 efficiency,
2 low-power efficiency) and the lowest CPU of its physical core. Every
CPU a performance core of its own where nothing is known. */
void janas_os_cpu_kinds(const int *cand, int nc, int *kind, int *leader);
#endif