Skip to content

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

968 B
Raw Blame
total_gb	trunk_gb	cache_gb	s_per_tok	hit_pct	gb_read	peak_rss	ids
32	2.7	24.3	31.23	0.0	25.83	32.01	17374,20829,10,427,414,1008,606,142957
32	6.8	20.2	30.58	0.0	25.83	31.79	17374,20829,10,427,414,1008,606,142957
32	10.8	16.2	31.25	0.0	25.83	31.24	17374,20829,10,427,414,1008,606,142957
32	16.2	10.8	30.88	0.0	25.83	31.9	17374,20829,10,427,414,1008,606,142957
32	21.6	5.4	29.13	0.0	25.83	32.13	17374,20829,10,427,414,1008,606,142957
32	25.6	1.4	28.06	0.0	25.83	32.02	17374,20829,10,427,414,1008,606,142957
128	12.3	110.7	28.38	44.02	14.46	127.89	17374,20829,10,427,414,1008,606,142957
128	30.8	92.2	25.2	42.93	14.74	127.53	17374,20829,10,427,414,1008,606,142957
128	49.2	73.8	25.69	33.97	17.06	128.14	17374,20829,10,427,414,1008,606,142957
128	73.8	49.2	18.37	32.2	17.51	128.19	17374,20829,10,427,414,1008,606,142957
128	98.4	24.6	19.46	0.0	25.83	127.36	17374,20829,10,427,414,1008,606,142957
128	110.0	13.0	16.8	0.0	25.83	127.85	17374,20829,10,427,414,1008,606,142957