Skip to content

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

668 B
Raw Blame
flowchart LR
    H["h in"] --> A1["aggregate over<br/>earlier blocks"]
    A1 --> ATT["attention<br/>KDA or MLA"]
    ATT --> A2["aggregate again<br/>no guard"]
    A2 --> M["MoE<br/>or dense FFN"]
    M --> HO["h out"]
    A1 -.->|"every 12 layers"| SNAP["snapshot<br/>and clear"]
    style H fill:#E4ECF3,stroke:#5C7C9B,color:#3B372F
    style A1 fill:#EAE4F1,stroke:#7E6E9C,color:#3B372F
    style ATT fill:#F2E9D4,stroke:#C0913E,color:#3B372F
    style A2 fill:#EAE4F1,stroke:#7E6E9C,color:#3B372F
    style M fill:#F4E7D3,stroke:#C07A42,color:#3B372F
    style HO fill:#E7ECDD,stroke:#6E9160,color:#3B372F
    style SNAP fill:#DDEBE7,stroke:#4E8E86,color:#3B372F