Skip to content

Open-source State-of-the-art TTS model which runs on a CPU 😻

Python

Merge pull request #173 from stellon-admin/mlx

Kitten0e6b2a7f9cfd
diff --cc README.md
index 24c3cd0,6d0eeca..a896ddd
--- a/README.md
+++ b/README.md
@@@ -211,12 -213,27 +213,30 @@@ print(model.last_report) # Generation
> Kitten-TTS 2 C++ is in beta. Please report any issues you encounter.
+ ## Running on a Mac
+
+ On an Apple-silicon Mac, install the optional MLX backend and select it:
+
+ ```bash
+ pip install "kittenml[mlx]"
+ ```
+
+ ```python
+ from kittenml import KittenTTS
+
+ m = KittenTTS("KittenML/kitten-tts-2", backend="mlx")
+ m.generate_to_file("One day, a little girl named Lily found a needle in her room.",
+ "hello.wav", voice="Bruno")
+ ```
+
+ See the [MLX guide](docs/mlx.md) for benchmarks and the full details.
+
## Running with vLLM
-For GPU inference with vLLM on Linux and an NVIDIA CUDA GPU, install the optional backend:
+For GPU inference with vLLM on Linux and an NVIDIA CUDA GPU with compute capability
+8.0 or newer (Ampere or later, for bfloat16), install the optional backend. GPUs with compute
+capability 7.5 can use a float16 fallback that is supported but not the optimal path
+([vLLM guide](docs/vllm.md#older-gpus-compute-capability-75)).
```bash
pip install "kittenml[vllm]"
diff --cc kittenml/get_model.py
index 840de8e,7f251dd..15b6a10
--- a/kittenml/get_model.py
+++ b/kittenml/get_model.py
@@@ -71,13 -72,15 +72,18 @@@ class KittenTTS(KittenTTSOnnx)
if model_type == KITTEN2_TYPE:
from .kittentts2 import KittenTTS2
if backend == "vllm":
- from .kittentts2.vllm.backend import validate_environment
- validate_environment(device)
+ from .kittentts2.vllm.backend import check_options, validate_environment
+ validate_environment(device, dict(vllm_options or {}).get("dtype"))
+ # Before the download: an unsupported GPU or a misspelled option
+ # should not cost gigabytes.
+ check_options(vllm_options, weights or config.get("default_weights", "packed"))
elif vllm_options is not None:
raise ValueError("vllm_options requires backend='vllm'")
+ if backend == "mlx":
+ from .kittentts2.mlx.backend import validate_environment
+ validate_environment(device)
+ elif mlx_options is not None:
+ raise ValueError("mlx_options requires backend='mlx'")
if local_dir is None:
local_dir = snapshot_download(
repo_id=repo_id, cache_dir=cache_dir, token=hf_token,