A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
C avx2c99cpu-inferencedeep-learningfrom-scratchinference-enginekimi-k3linear-attentionllmllm-inferencemachine-learningmemory-efficientmixture-of-expertsmoemxfp4quantizationsimdsystems-programmingtransformerzero-dependencies
flowchart LR
H["h in"] --> A1["aggregate over<br/>earlier blocks"]
A1 --> ATT["attention<br/>KDA or MLA"]
ATT --> A2["aggregate again<br/>no guard"]
A2 --> M["MoE<br/>or dense FFN"]
M --> HO["h out"]
A1 -.->|"every 12 layers"| SNAP["snapshot<br/>and clear"]
style H fill:#E4ECF3,stroke:#5C7C9B,color:#3B372F
style A1 fill:#EAE4F1,stroke:#7E6E9C,color:#3B372F
style ATT fill:#F2E9D4,stroke:#C0913E,color:#3B372F
style A2 fill:#EAE4F1,stroke:#7E6E9C,color:#3B372F
style M fill:#F4E7D3,stroke:#C07A42,color:#3B372F
style HO fill:#E7ECDD,stroke:#6E9160,color:#3B372F
style SNAP fill:#DDEBE7,stroke:#4E8E86,color:#3B372F