A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
Measurement data
The raw output behind every number in PERFORMANCE.md and TUNING.md. Each table there cites the file it comes from.
Everything was produced in one campaign on one machine. See environment.txt for the full capture, including the measurement hygiene applied before any timing was taken. Summary: AMD EPYC 7763, 124 vCPU (2×62, no SMT), 228 GiB RAM, 3.2 TB NVMe at 3.2 GB/s O_DIRECT, Ubuntu 24.04.3, GCC 13.3, AVX2 without AVX-512.
| file | what it is | backs |
|---|---|---|
| memory-ladder.tsv | 12 memory budgets, each under a hard cgroup cap | the ladder table, the two-cache finding |
| trunk-cache-split.tsv | 12 splits at two fixed budgets (32 GB, 128 GB) | the allocation recommendation |
| replication.tsv | 3 runs of one identical configuration | the noise floor |
| expert-cache-capacity.txt | LRU / Belady / pinned hit rates from the recorded trace | why the expert cache is weak |
| trunk-quantisation.txt | int8 vs int4 reconstruction error on real weights | why the trunk is streamed, not quantised |
| generations.txt | what the model actually produced, decoded | the output claims |
| environment.txt | the machine | everything |
Read the noise floor first
run s/token
1 14.7760
2 14.6655
3 20.1408
Same binary, same prompt, same flags, same minute. mean 16.53, sd 3.13, spread 33.1%.
A third of the measurement is noise. Every single-sample comparison in these files is subject to it, and any difference smaller than ~33% between two single runs is not an established effect. This is stated up front because it invalidated several conclusions drawn earlier in the campaign, including a "128 GB outlier" that was simply dispersion.
What the split data actually shows
This is the one place where the raw data is stronger than a naive reading and weaker than a careless one, so it is worth being exact.
128 GB, six splits (trunk GB → s/token):
12.3 → 28.38 30.8 → 25.20 49.2 → 25.69
73.8 → 18.37 98.4 → 19.46 110.0 → 16.80
- Endpoint ratio 1.689×, which is where the 1.69× figure comes from.
- Not monotonic. Two inversions: +1.9% at 30.8→49.2, and +5.9% at 73.8→98.4.
- Spearman ρ = −0.886 over the six points.
32 GB, six splits, an independent replication of the same question at a quarter of the memory:
2.7 → 31.23 6.8 → 30.58 10.8 → 31.25
16.2 → 30.88 21.6 → 29.13 25.6 → 28.06
- Endpoint ratio 1.113×. One inversion, +2.2%.
- Spearman ρ = −0.714.
In summary: twelve points across two independent budgets, both with a strong negative rank correlation between trunk share and time per token. Every inversion is far inside the 33% noise floor, so none of them is evidence against the trend, and none of the individual steps is evidence for it either. What carries the conclusion is the rank correlation across twelve points and the mechanism, the trunk is re-read in full every token (108.81 GB) while only ~25.8 GB of experts are touched.
What this data does not establish: that 1.69× is the effect size. It is one endpoint
comparison of single samples. The direction is well supported; the magnitude is not
replicated. benchmarks/split-sweep.sh re-runs the experiment with repetitions.
Reproducing
memory-ladder.tsv and trunk-cache-split.tsv are regenerated by
benchmarks/memory-ladder.sh and benchmarks/split-sweep.sh respectively. Both need the
1.56 TB checkpoint, a packed trunk, and a Linux host with systemd (for the cgroup caps
that make a memory budget mean anything). Both write a machine.txt alongside their
results so a new run can be compared against this one.
expert-cache-capacity.txt needs neither weights nor a GPU, it replays
tests/fixtures/expert_trace.bin, which ships in this repository:
python3 tools/sim_cache.py tests/fixtures/expert_trace.bin
That is the one result here anyone can re-derive in seconds, and it is the one that explains the two-cache finding.
trunk-quantisation.txt sampled the released checkpoint over HTTP range reads rather
than downloading it. The script that did so is not shipped; the file records its method
and its limits.
Scope
These are engine measurements: speed, memory, bytes moved, cache behaviour, and what the
model emitted. They are not quality benchmarks. No perplexity, no task evaluation, no
comparison against another implementation's output quality. The correctness evidence is
separate and stronger: see ../TESTING.md and the logs in
tests/fixtures/gates/ and tests/fixtures/golden/.