Skip to content

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

2.6 KB
Raw Blame

Documentation

Start here

QUICKSTART.md from nothing to generated text
TUNING.md choosing a memory budget, one decision dominates

Reference

ARCHITECTURE.md how the model maps onto the code
API.md the C interface, for embedding the engine
PERFORMANCE.md the memory ladder, measured, with its noise floor
TESTING.md what each gate proves
BENCHMARKING.md how to measure without fooling yourself
ROADMAP.md what is missing, in priority order
data/ the raw measurement output every table is transcribed from

Text-only official XTML chat is documented in the main README. It remains CPU- and disk-bound and uses the same streamed trunk and expert-cache budgets as batch generation.

Upstream material

kimi-k3-tech-report.pdf the model's technical report, as published

Reproduced for reference; it remains the property of its authors. Every architectural claim in ARCHITECTURE.md cites the section of this report it comes from, so the two can be checked against each other.

The three claims worth checking first

If you are evaluating whether this project is worth your time, these are the load-bearing claims and where each is substantiated:

  1. A 2.78T-parameter model runs in 8 GB of RAM. PERFORMANCE.md, twelve budgets, measured peak RSS at each. Raw: data/memory-ladder.tsv.

  2. Output is byte-identical across memory budgets. Memory buys speed, not capability. PERFORMANCE.md, and examples/02-memory-budgets.sh reproduces it in three runs. The ids column of data/memory-ladder.tsv is the claim in raw form: twelve rows, one identical token sequence.

  3. The engine matches a reference implementation exactly. Teacher forcing, greedy decode, and incremental decode, on a model with the released tensor graph. TESTING.md; make test runs it in seconds with no weights.

Claims about speed carry a caveat that PERFORMANCE.md states up front: run-to-run variance on identical configurations is 33% (data/replication.tsv), so single-sample differences below that are not effects. Claims about output carry no such caveat, those are exact, and they are counts rather than stopwatch readings.