A research project focused on developing a lightweight neural voice cloning and speech synthesis system, optimized for deployment on edge devices.
Reference
Model variants
safetensors
PyTorch weights for the reverb-tts package, in four variants (TTS.from_pretrained(variant=...)):
| Variant | Weights | Activations | Download |
|---|---|---|---|
fp32 |
fp32 | full precision | 1.19 GB |
fp16 |
fp16 | full precision | 594 MB |
int8 (default) |
int8, one scale per output channel | full precision | 307 MB |
int4 |
int4, one scale per group of 32 inputs | full precision | 191 MB |
In the quantized variants the decoder's linear layers hold the quantized weights, with fp16 scales. On an internal test
set, int8 showed no quality loss against fp32, and int4 none that we could measure. The benchmarks use full
precision.
ExecuTorch
ExecuTorch programs for on-device inference, with the codec kept out of the program:
| Program | Hardware | Size |
|---|---|---|
| MLX fp16 | Apple GPUs | 612 MB |
| MLX int8 | Apple GPUs | 352 MB |
| MLX int4 | Apple GPUs | 232 MB |
| Vulkan int4 | Android GPUs | 233 MB |
| QNN int4 | Qualcomm NPUs, one program per chip: Snapdragon 8 Gen 1 (SM8450, v69), 8 Gen 2 (SM8550, v73), 8 Gen 3 (SM8650, v75), 8 Elite (SM8750, v79), 8 Elite Gen 5 (SM8850, v81) | 232–237 MB each |
The programs take text exactly as given, so normalize it first, as the package does: we strongly recommend Verbalis, which has a C++ API for on-device apps.
In the browser
ruNNtime runs Reverb on the GPU in the browser through WebGPU, with the weights from the Hugging Face Hub.
Repository layout
This repository:
reverb_tts/
tts.py TTS and Voice: the public API
text.py text preparation, length estimate, long-text splitting
audio.py loading audio, trimming silence
sampler.py the sampler and its settings
model.py the network
tests/ tests on a small random model, no download needed
docs/
The weights on Hugging Face, software-mansion/Reverb-TTS:
model/
tokenizer.json, tokenizer_config.json, chat_template.jinja
safetensors/<variant>/ config.json, model.safetensors: fp32, fp16, int8, int4
executorch/mlx/<precision>/ reverb_mlx_<precision>.pte: fp16, int8, int4
executorch/vulkan/int4/ reverb_vulkan_int4.pte
executorch/qnn/int4/ reverb_qnn_int4_<hexagon>.pte: v69, v73, v75, v79, v81
runntime/<precision>/ model.safetensors for ruNNtime: int8, int4
voices/ preset voices: <name>.json
assets/ images for the model card