A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
C avx2c99cpu-inferencedeep-learningfrom-scratchinference-enginekimi-k3linear-attentionllmllm-inferencemachine-learningmemory-efficientmixture-of-expertsmoemxfp4quantizationsimdsystems-programmingtransformerzero-dependencies
total_gb trunk_gb cache_gb s_per_tok hit_pct gb_read peak_rss ids
32 2.7 24.3 31.23 0.0 25.83 32.01 17374,20829,10,427,414,1008,606,142957
32 6.8 20.2 30.58 0.0 25.83 31.79 17374,20829,10,427,414,1008,606,142957
32 10.8 16.2 31.25 0.0 25.83 31.24 17374,20829,10,427,414,1008,606,142957
32 16.2 10.8 30.88 0.0 25.83 31.9 17374,20829,10,427,414,1008,606,142957
32 21.6 5.4 29.13 0.0 25.83 32.13 17374,20829,10,427,414,1008,606,142957
32 25.6 1.4 28.06 0.0 25.83 32.02 17374,20829,10,427,414,1008,606,142957
128 12.3 110.7 28.38 44.02 14.46 127.89 17374,20829,10,427,414,1008,606,142957
128 30.8 92.2 25.2 42.93 14.74 127.53 17374,20829,10,427,414,1008,606,142957
128 49.2 73.8 25.69 33.97 17.06 128.14 17374,20829,10,427,414,1008,606,142957
128 73.8 49.2 18.37 32.2 17.51 128.19 17374,20829,10,427,414,1008,606,142957
128 98.4 24.6 19.46 0.0 25.83 127.36 17374,20829,10,427,414,1008,606,142957
128 110.0 13.0 16.8 0.0 25.83 127.85 17374,20829,10,427,414,1008,606,142957