Open-source State-of-the-art TTS model which runs on a CPU π»
Architecture
KittenTTS 2 is not a phoneme-to-waveform model. It is a language model over a vocabulary shared between text and audio: it reads your text and writes S3 audio codec tokens, which the s3gen vocoder turns into 24 kHz speech. Voice identity enters through a speaker embedding projected into the first position's hidden state, plus the reference clip's own codec tokens in the prompt.
Package layout
kittenml/
get_model.py KittenTTS(...) -- reads config.json, builds the right backend
repo.py repository resolution shared by both backends
preprocess.py text normalization shared by both backends
kittentts2/ the KittenTTS 2 speech language model
model.py KittenTTS2: generate, clone, streaming
prompt.py reference prefix and target segment assembly
logits.py decode-time sampling and runaway penalties
speaker.py spk_proj head, pyannote embedding, Whisper transcription
text.py expression markup, normalization, chunking, joining
tokens.py the shared text/audio token layout
vocoder.py reference encoding and token -> waveform
s3gen.py vendored S3 tokenizer + s3gen vocoder
mlx/ the optional Apple-silicon backend
loader.py ternary weights -> MLX's 4-bit kernels, exactly
model.py Qwen3 and the speaker head as MLX modules
sample.py the same decode pipeline, in MLX
kittentts_legacy/ the ONNX models
model.py KittenTTSOnnx: generate, speed control
onnx_model.py the ONNX session and phonemizer
Neither backend is imported until it is the one selected, so a KittenTTS 2 install never needs
espeak and an ONNX install never needs torch. The same is true of the optional language-model
backends under kittentts2/: mlx/ is imported only for backend="mlx", vllm/ only for
backend="vllm".
The model repository
A KittenTTS 2 repository is assembled by internal tooling from a training bundle, and holds everything inference needs:
config.json token layout, decode presets, voice and decoder indexes
lm/ the speech language model, plus its spk_proj head
speaker/ the speaker-embedding model and its license
voices/ reference clips, transcripts, and precomputed embeddings
decoders/ optional quantised decoders
Precomputing the per-voice embeddings is what lets a built-in voice skip the embedding model entirely at run time. Nothing in the repository is gated, so installing KittenTTS 2 needs no Hugging Face credentials.
Only the paths config.json names are downloaded, so a repository can carry artifacts for other
runtimes without every install paying for them.
Packed language-model weights
The LM's linear weights are ternary: within each 128-wide group every value is exactly one of
{-scale, 0, +scale}. Stored as bf16 that spends 16 bits per weight to express one of three
states, so ternary.py packs them five trits to a byte β 1.6 bits each, log2(3) rounded up to a
byte boundary β keeping every group's scale at full bf16 precision. The token embedding, layer
norms and speaker head are not ternary and are stored unchanged.
That takes the checkpoint from 3.47 GB to 0.95 GB losslessly: unpacking reproduces the
original tensors bit-for-bit, and generated audio is bit-identical. config.json advertises it as
lm_packed, and the loader prefers it when present.
See example.py for a runnable script, and example_cloning.py, example_expression.py, example_streaming.py and example_advanced.py for the rest of the API.