Open-source State-of-the-art TTS model which runs on a CPU 😻
Python
Serve a basic OpenAI-compatible speech API from the Docker image
haaziq-sl13628c1f9e77
diff --git a/README.md b/README.mdindex a9cfc5a..5623cf0 100644--- a/README.md+++ b/README.md@@ -253,7 +253,7 @@ Two consequences worth knowing:| [Decoders](docs/decoders.md) | How audio is decoded, and the smaller quantised decoders || [Text normalization](docs/text-normalization.md) | How written text becomes spoken text || [Architecture](docs/architecture.md) | What the model is, package layout, vendored components |-| [Docker](docs/docker.md) | Run the vLLM or PyTorch GPU backend in a container |+| [Docker](docs/docker.md) | Run a basic OpenAI-compatible speech API, with voice cloning, in a container || [Lightweight ONNX models](docs/onnx-models.md) | The CPU models, 15M-80M parameters, and their API |## System Requirementsdiff --git a/docs/README.md b/docs/README.mdindex 86473e7..d5f51b5 100644--- a/docs/README.md+++ b/docs/README.md@@ -8,7 +8,7 @@ Fuller detail than the [project README](../README.md), which covers getting star| [voices-and-expression.md](voices-and-expression.md) | The 47 built-in voices, the emotion and vocal-event tags, the ten languages || [decoders.md](decoders.md) | How audio is decoded, and the smaller quantised decoders || [vllm.md](vllm.md) | Optional NVIDIA GPU inference with vLLM |-| [docker.md](docker.md) | Running the vLLM or PyTorch GPU backend in a container |+| [docker.md](docker.md) | A basic OpenAI-compatible speech API, with voice cloning, in a container || [text-normalization.md](text-normalization.md) | How written text becomes spoken text || [architecture.md](architecture.md) | What the model is, package layout, vendored components || [onnx-models.md](onnx-models.md) | The lightweight CPU models and their API |diff --git a/docs/docker.md b/docs/docker.mdindex 2e66131..6c36171 100644--- a/docs/docker.md+++ b/docs/docker.md@@ -1,27 +1,104 @@# Running with Docker+The image runs a basic OpenAI-compatible speech API on one NVIDIA GPU: speech as a+complete file or streamed, and voice cloning. Its routes follow the+[hosted KittenML API](https://docs.kittenml.com/tts/generate-speech).+Use a Linux host with Docker, an NVIDIA GPU and the[NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).The host driver must meet the [vLLM GPU requirements](https://docs.vllm.ai/en/v0.31.0/getting_started/installation/gpu/).-From the repository root, build the image:+## Start the server++Build the image from the root of a KittenTTS checkout. The final `.` is that+directory, which the build reads the package from:++```sh+git clone https://github.com/KittenML/KittenTTS.git+cd KittenTTS+docker build -f kittenml/kittentts2/docker/Dockerfile -t kittenml .+```++Then start the server, from any directory:++```sh+docker run --rm --gpus all --ipc=host -p 8000:8000 -v kittenml-cache:/cache kittenml+```++The first start downloads the model and compiles GPU kernels, which takes a few+minutes. The named volume keeps both, so later starts are faster. The server is+ready when it prints `Ready`. Model files are not included in the image.++## Generate speech++```sh+curl http://localhost:8000/v1/audio/speech \+ -H "Content-Type: application/json" \+ -d '{"model": "kitten-tts-2-latest", "input": "Hello from Kitten TTS.",+ "voice": "Bruno", "response_format": "wav", "stream": false}' \+ --output speech.wav+```++`"stream": false` returns the complete file. Without it, audio is streamed as each+sentence chunk is generated; `"stream_format": "sse"` sends Base64+`speech.audio.delta` events and a final `speech.audio.done`. `response_format` is+`mp3` (the default), `wav`, `pcm`, `flac` or `opus`; `flac` and `opus` are sent once+complete. `GET /v1/voices` lists the 47 built-in voices.++The OpenAI SDK works with any API key:++```python+from openai import OpenAI++client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")+with client.audio.speech.with_streaming_response.create(+ model="kitten-tts-2-latest", voice="Bruno", input="Hello there.",+ response_format="mp3") as response:+ response.stream_to_file("speech.mp3")+```++## Clone a voice++Upload 2-30 seconds of one speaker as `reference_audio` to clone it for one request:```sh-docker build -t kittenml .+curl http://localhost:8000/v1/audio/speech \+ -F model=kitten-tts-2-latest \+ -F input="This sentence uses the uploaded voice." \+ -F [email protected] \+ -F response_format=wav -F stream=false \+ --output cloned.wav```-Save the [README's vLLM example](../README.md#running-with-vllm) as-`vllm_example.py`, then run it:+Or save the voice once and reuse its ID as `"voice": {"id": "voice_..."}`:```sh-docker run --rm --gpus all --ipc=host \- -v kittenml-cache:/cache -v "$PWD:/workspace" \- kittenml python vllm_example.py+curl http://localhost:8000/v1/audio/voices \+ -F name="My voice" \+ -F [email protected]```-The command saves `output.wav` in the current directory. The named volume keeps-model downloads and compilation caches between runs. Model files are downloaded-on first use and are not included in the image.+Saved voices are kept in the cache volume and deleted with+`DELETE /v1/audio/voices/{voice_id}`. The clip is transcribed with Whisper, which+needs more GPU memory; add `-F reference_transcript="..."` with the exact words+spoken to skip it.-The image also includes the existing PyTorch backend. Run your own Python script-by replacing `vllm_example.py` in the command above.+## Options++Pass these with `-e`:++| Variable | Default | Meaning |+|---|---|---|+| `KITTENML_BACKEND` | `vllm` | `torch` uses the PyTorch backend |+| `KITTENML_VLLM_OPTIONS` | | `vllm_options` as JSON, e.g. `{"gpu_memory_utilization": 0.7}` on an 8 GB GPU |++The server has no authentication, so do not expose its port publicly. Unlike the+hosted API, it supports only `speed` 1.0, uses the voice names from `/v1/voices`,+and has no consent resources.++To run your own script instead, run this from the directory that contains it:++```sh+docker run --rm --gpus all --ipc=host -v kittenml-cache:/cache -v "$PWD:/workspace" \+ kittenml python my_script.py+```diff --git a/Dockerfile b/kittenml/kittentts2/docker/Dockerfilesimilarity index 70%rename from Dockerfilerename to kittenml/kittentts2/docker/Dockerfileindex 431ebfb..8ce2812 100644--- a/Dockerfile+++ b/kittenml/kittentts2/docker/Dockerfile@@ -2,7 +2,9 @@ FROM python:3.12-slim-bookwormENV PYTHONUNBUFFERED=1 \HF_HOME=/cache/huggingface \- VLLM_CACHE_ROOT=/cache/vllm+ VLLM_CACHE_ROOT=/cache/vllm \+ CUDA_CACHE_PATH=/cache/nv \+ KITTENML_VOICES_DIR=/cache/voices# Triton compiles small GPU kernels at runtime and needs a C compiler.RUN apt-get update \@@ -15,3 +17,4 @@ COPY kittenml ./kittenmlRUN pip install --no-cache-dir ".[vllm]"WORKDIR /workspace+CMD ["python", "-m", "kittenml.kittentts2.docker.server", "--host", "0.0.0.0"]diff --git a/kittenml/kittentts2/docker/server.py b/kittenml/kittentts2/docker/server.pynew file mode 100644index 0000000..27917be--- /dev/null+++ b/kittenml/kittentts2/docker/server.py@@ -0,0 +1,453 @@+"""A basic OpenAI-compatible speech API for KittenTTS 2.++Serves a subset of the hosted KittenML API from one local model:++ POST /v1/audio/speech speech as a complete file, streamed audio or SSE,+ optionally cloned from reference audio+ POST /v1/audio/voices save a cloned voice+ GET /v1/voices list built-in and saved voices+ DELETE /v1/audio/voices/{id} delete a saved voice+ GET /health++Usage: python -m kittenml.kittentts2.docker.server [--host 0.0.0.0] [--port 8000].+KITTENML_BACKEND selects "vllm" (default) or "torch"; KITTENML_VLLM_OPTIONS takes+vllm_options as JSON; KITTENML_VOICES_DIR is where saved voices are kept. See+docs/docker.md.+"""+import argparse+import asyncio+import base64+import binascii+import concurrent.futures+import functools+import io+import json+import os+import pathlib+import re+import shutil+import struct+import tempfile+import time+import traceback+import uuid+from typing import Literal++import numpy as np+import soundfile as sf+import uvicorn+from fastapi import FastAPI, Request+from fastapi.responses import JSONResponse, Response, StreamingResponse+from pydantic import BaseModel, ConfigDict, Field, ValidationError++from kittenml import KittenTTS++MODEL_IDS = ("kitten-tts-2-latest", "kitten-tts-2")+MAX_UPLOAD_BYTES = 10 * 1024 * 1024+REFERENCE_SECONDS = (2.0, 30.0)+AUDIO_SUFFIXES = (".wav", ".mp3", ".flac", ".ogg")+VOICE_ID = re.compile(r"voice_[0-9a-f]{32}")+# response_format -> (soundfile format, subtype, media type)+FORMATS = {+ "mp3": ("MP3", "MPEG_LAYER_III", "audio/mpeg"),+ "wav": ("WAV", "PCM_16", "audio/wav"),+ "flac": ("FLAC", "PCM_16", "audio/flac"),+ "opus": ("OGG", "OPUS", "audio/ogg"),+ "pcm": (None, None, "audio/pcm"),+}+# Formats that can be sent chunk by chunk; flac and opus are sent once complete.+PROGRESSIVE = ("mp3", "wav", "pcm")+++class SpeechRequest(BaseModel):+ model_config = ConfigDict(extra="forbid")++ model: str = MODEL_IDS[0]+ input: str = Field(min_length=1, max_length=40_000)+ voice: str | dict | None = None+ response_format: Literal["mp3", "wav", "flac", "opus", "pcm"] = "mp3"+ speed: float = 1.0+ stream: bool = True+ stream_format: Literal["audio", "sse"] = "audio"+ mode: Literal["stable", "expressive"] | None = None+ temperature: float | None = Field(None, ge=0.3, le=2.0)+ top_p: float | None = Field(None, ge=0.5, le=1.0)+ top_k: int | None = Field(None, ge=0, le=200)+ min_p: float | None = Field(None, ge=0.0, le=0.5)+ max_new_tokens: int | None = Field(None, ge=500, le=12_000)+++class APIError(Exception):+ def __init__(self, status, message, code="invalid_request_error", param=None):+ super().__init__(message)+ self.status, self.message, self.code, self.param = status, message, code, param+++def error_body(status, message, code, param=None):+ kind = "invalid_request_error" if status < 500 else "server_error"+ return {"error": {"message": message, "type": kind, "param": param, "code": code}}+++class Engine:+ """Holds the model. Every model call runs on one worker thread, one at a time."""++ def __init__(self, backend, vllm_options, voices_dir):+ self.backend = backend+ self.voices_dir = pathlib.Path(voices_dir)+ self.voices_dir.mkdir(parents=True, exist_ok=True)+ self.pool = concurrent.futures.ThreadPoolExecutor(max_workers=1)+ options = {"vllm_options": vllm_options} if vllm_options else {}+ self.model = self.pool.submit(+ KittenTTS, "KittenML/kitten-tts-2",+ backend="vllm" if backend == "vllm" else None, **options).result()++ async def run(self, fn, *args, **kwargs):+ loop = asyncio.get_running_loop()+ return await loop.run_in_executor(self.pool, functools.partial(fn, *args, **kwargs))++ def forget(self, reference, transcript, directory=None):+ """Drop a cloned reference from the model's cache, and its files if temporary."""+ self.model._reference_cache.pop(("clone", str(reference), transcript), None)+ if directory:+ shutil.rmtree(directory, ignore_errors=True)+++ENGINE: Engine = None+app = FastAPI(title="KittenTTS 2", docs_url=None, redoc_url=None)++[email protected]_handler(APIError)+async def _api_error(request, exc):+ return JSONResponse(error_body(exc.status, exc.message, exc.code, exc.param), exc.status)++[email protected]_handler(Exception)+async def _server_error(request, exc):+ traceback.print_exception(exc)+ return JSONResponse(error_body(500, "internal server error", "server_error"), 500)+++def parse_request(fields):+ try:+ return SpeechRequest.model_validate(fields)+ except ValidationError as exc:+ first = exc.errors()[0]+ param = ".".join(str(p) for p in first["loc"]) or None+ raise APIError(400, f"{param}: {first['msg']}" if param else first["msg"],+ param=param) from None+++def save_reference(data, suffix, param):+ """Write uploaded reference audio to a temporary directory and check it."""+ if len(data) > MAX_UPLOAD_BYTES:+ raise APIError(413, "reference audio is larger than 10 MiB", "request_too_large", param)+ suffix = suffix.lower() if suffix.lower() in AUDIO_SUFFIXES else ".wav"+ directory = tempfile.mkdtemp(prefix="kittenml-ref-")+ path = pathlib.Path(directory) / f"reference{suffix}"+ path.write_bytes(data)+ try:+ info = sf.info(str(path))+ seconds = info.frames / info.samplerate+ except Exception:+ shutil.rmtree(directory, ignore_errors=True)+ raise APIError(400, "reference audio must be WAV, MP3, FLAC or OGG",+ "invalid_audio", param) from None+ if not REFERENCE_SECONDS[0] <= seconds <= REFERENCE_SECONDS[1]:+ shutil.rmtree(directory, ignore_errors=True)+ raise APIError(400, "reference audio must be 2 to 30 seconds long",+ "invalid_audio_duration", param)+ return path, directory+++async def read_upload(upload, param):+ if upload is None or isinstance(upload, str):+ raise APIError(400, f"{param} must be an uploaded file", param=param)+ data = await upload.read(MAX_UPLOAD_BYTES + 1)+ return save_reference(data, pathlib.Path(upload.filename or "").suffix, param)+++def load_saved_voice(voice_id):+ meta = ENGINE.voices_dir / voice_id / "voice.json"+ if not VOICE_ID.fullmatch(voice_id) or not meta.is_file():+ raise APIError(404, f"voice {voice_id!r} not found", "voice_not_found", "voice")+ return json.loads(meta.read_text())+++def resolve_voice(voice):+ """Speaker arguments for generate(), and a temporary directory to remove afterwards."""+ if voice is None:+ return {}, None+ if isinstance(voice, dict) and set(voice) == {"id"}:+ voice = voice["id"]+ if isinstance(voice, str):+ if voice.startswith("voice_"):+ saved = load_saved_voice(voice)+ return {"reference": saved["reference"],+ "reference_text": saved["reference_transcript"]}, None+ if voice not in ENGINE.model.available_voices:+ raise APIError(400, f"unknown voice {voice!r}; GET /v1/voices lists them",+ param="voice")+ return {"voice": voice}, None+ inline = voice.get("reference_audio") if isinstance(voice, dict) else None+ if not isinstance(inline, dict) or not set(voice) <= {"reference_audio",+ "reference_transcript"}:+ raise APIError(400, "voice must be a voice name, {\"id\": \"voice_...\"} or "+ "{\"reference_audio\": {\"data\", \"format\"}}", param="voice")+ try:+ data = base64.b64decode(inline.get("data", ""), validate=True)+ except (binascii.Error, TypeError):+ raise APIError(400, "reference_audio.data must be Base64",+ param="voice.reference_audio") from None+ path, directory = save_reference(data, "." + str(inline.get("format", "wav")),+ "voice.reference_audio")+ return {"reference": str(path),+ "reference_text": voice.get("reference_transcript") or None}, directory+++def encode(audio, response_format, sample_rate):+ audio = np.clip(np.asarray(audio, dtype=np.float32), -1.0, 1.0)+ if response_format == "pcm":+ return (audio * 32767).astype("<i2").tobytes()+ container, subtype, _ = FORMATS[response_format]+ # MP3 at a constant 96 kbps, so streamed pieces join into one readable stream.+ options = {"bitrate_mode": "CONSTANT", "compression_level": 0.4} \+ if container == "MP3" else {}+ buffer = io.BytesIO()+ sf.write(buffer, audio, sample_rate, format=container, subtype=subtype, **options)+ return buffer.getvalue()+++# Layer III bitrates for MPEG-2/2.5 and MPEG-1, and sample rates per MPEG version.+MP3_KBPS = ((0, 8, 16, 24, 32, 40, 48, 56, 64, 80, 96, 112, 128, 144, 160),+ (0, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320))+MP3_RATES = {3: (44100, 48000, 32000), 2: (22050, 24000, 16000), 0: (11025, 12000, 8000)}+++def without_mp3_tag(data):+ """Drop the encoder's leading Xing/Info frame. Its frame count covers one piece only,+ so decoders would stop after the first piece of a streamed response."""+ try:+ version = (data[1] >> 3) & 3 # 3 = MPEG-1, 2 = MPEG-2, 0 = MPEG-2.5+ kbps = MP3_KBPS[version == 3][data[2] >> 4]+ rate = MP3_RATES[version][(data[2] >> 2) & 3]+ size = (144 if version == 3 else 72) * kbps * 1000 // rate + ((data[2] >> 1) & 1)+ except (IndexError, KeyError):+ return data+ frame = data[:size]+ return data[size:] if b"Xing" in frame or b"Info" in frame else data+++def streaming_wav_header(sample_rate):+ # Unknown length: players read PCM until the connection closes.+ return (b"RIFF" + struct.pack("<I", 0xFFFFFFFF) + b"WAVEfmt "+ + struct.pack("<IHHIIHH", 16, 1, 1, sample_rate, sample_rate * 2, 2, 16)+ + b"data" + struct.pack("<I", 0xFFFFFFFF))+++def sse_event(payload):+ return f"data: {json.dumps(payload)}\n\n".encode()++[email protected]("/health")+async def health():+ return {"status": "ok", "backend": ENGINE.backend, "sample_rate": ENGINE.model.sample_rate}++[email protected]("/v1/audio/speech")+async def speech(request: Request):+ temporary = None+ if request.headers.get("content-type", "").startswith("multipart/form-data"):+ form = await request.form()+ fields = {}+ for key, value in form.multi_items():+ if key in fields:+ raise APIError(400, f"duplicate field {key!r}", param=key)+ fields[key] = value+ if "voice" in fields:+ raise APIError(400, "multipart requests take reference_audio instead of voice",+ param="voice")+ upload = fields.pop("reference_audio", None)+ if upload is None:+ raise APIError(400, "multipart speech requests need a reference_audio file; "+ "send JSON for a built-in voice", param="reference_audio")+ transcript = fields.pop("reference_transcript", None) or None+ req = parse_request(fields)+ path, temporary = await read_upload(upload, "reference_audio")+ speaker = {"reference": str(path), "reference_text": transcript}+ else:+ try:+ body = await request.json()+ except ValueError:+ raise APIError(400, "send a JSON object, or multipart/form-data with "+ "reference_audio") from None+ if not isinstance(body, dict):+ raise APIError(400, "the JSON body must be an object")+ req = parse_request(body)+ speaker, temporary = resolve_voice(req.voice)++ try:+ if req.model not in MODEL_IDS:+ raise APIError(400, f"model must be {MODEL_IDS[0]!r}", "model_not_found", "model")+ if req.speed != 1.0:+ raise APIError(400, "this server supports speed 1.0 only", param="speed")+ if not req.stream and req.stream_format == "sse":+ raise APIError(400, "\"stream\": false cannot be combined with "+ "stream_format \"sse\"", param="stream_format")+ except APIError:+ if temporary:+ shutil.rmtree(temporary, ignore_errors=True)+ raise++ options = dict(speaker, preset=req.mode, temperature=req.temperature, top_p=req.top_p,+ top_k=req.top_k, min_p=req.min_p)+ if req.max_new_tokens:+ options["max_new_tokens"] = req.max_new_tokens++ def cleanup():+ # Only one-request clones are forgotten; saved voices stay cached.+ if temporary:+ ENGINE.forget(options["reference"], options["reference_text"], temporary)++ rate = ENGINE.model.sample_rate+ media_type = FORMATS[req.response_format][2]+ progressive = req.stream and req.response_format in PROGRESSIVE++ if not progressive:+ # The complete file: generate() joins the chunks and smooths the seams.+ try:+ audio = await ENGINE.run(ENGINE.model.generate, req.input, **options)+ except ValueError as exc:+ raise APIError(400, str(exc)) from None+ finally:+ ENGINE.pool.submit(cleanup)+ data = encode(audio, req.response_format, rate)+ if req.stream_format != "sse":+ return Response(data, media_type=media_type)+ events = [sse_event({"type": "speech.audio.delta",+ "audio": base64.b64encode(data).decode()}),+ sse_event({"type": "speech.audio.done",+ "usage": usage(len(audio) / rate, req.input)})]+ return StreamingResponse(iter(events), media_type="text/event-stream")++ # Streaming: one piece per sentence chunk, as generate_stream yields them. The first+ # chunk is produced before responding, so a failure still returns a proper error.+ chunks = ENGINE.model.generate_stream(req.input, **options)+ try:+ first = await ENGINE.run(next, chunks, None)+ except ValueError as exc:+ ENGINE.pool.submit(chunks.close)+ ENGINE.pool.submit(cleanup)+ raise APIError(400, str(exc)) from None+ except BaseException:+ ENGINE.pool.submit(chunks.close)+ ENGINE.pool.submit(cleanup)+ raise+ if first is None:+ ENGINE.pool.submit(cleanup)+ raise APIError(500, "the model produced no audio", "server_error")++ async def body():+ wav, seconds, index = first, 0.0, 0+ try:+ while wav is not None:+ if req.response_format == "wav":+ # Each chunk as raw PCM after one streaming header.+ piece = ((streaming_wav_header(rate) if index == 0 else b"")+ + encode(wav, "pcm", rate))+ elif req.response_format == "mp3":+ piece = without_mp3_tag(encode(wav, "mp3", rate))+ else:+ piece = encode(wav, req.response_format, rate)+ seconds += len(wav) / rate+ index += 1+ if req.stream_format == "sse":+ yield sse_event({"type": "speech.audio.delta",+ "audio": base64.b64encode(piece).decode()})+ else:+ yield piece+ wav = await ENGINE.run(next, chunks, None)+ if req.stream_format == "sse":+ yield sse_event({"type": "speech.audio.done",+ "usage": usage(seconds, req.input)})+ finally:+ # Queued behind any chunk still being generated for this request.+ ENGINE.pool.submit(chunks.close)+ ENGINE.pool.submit(cleanup)++ media = "text/event-stream" if req.stream_format == "sse" else media_type+ return StreamingResponse(body(), media_type=media)+++def usage(seconds, text):+ return {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0,+ "audio_seconds": round(seconds, 2), "input_characters": len(text)}++[email protected]("/v1/audio/voices")+async def create_voice(request: Request):+ form = await request.form()+ unknown = set(form.keys()) - {"name", "audio_sample", "reference_transcript"}+ if unknown:+ raise APIError(400, f"unknown field {sorted(unknown)[0]!r}", param=sorted(unknown)[0])+ name = form.get("name")+ if not isinstance(name, str) or not 1 <= len(name.strip()) <= 256:+ raise APIError(400, "name must be 1 to 256 characters", param="name")+ transcript = form.get("reference_transcript")+ if transcript is not None and not isinstance(transcript, str):+ raise APIError(400, "reference_transcript must be text", param="reference_transcript")+ path, temporary = await read_upload(form.get("audio_sample"), "audio_sample")+ voice_id = f"voice_{uuid.uuid4().hex}"+ folder = ENGINE.voices_dir / voice_id+ folder.mkdir()+ reference = folder / f"reference{path.suffix}"+ shutil.move(str(path), reference)+ shutil.rmtree(temporary, ignore_errors=True)+ voice = {"id": voice_id, "object": "audio.voice", "name": name.strip(),+ "created_at": int(time.time())}+ (folder / "voice.json").write_text(json.dumps(+ {**voice, "reference": str(reference),+ "reference_transcript": (transcript or "").strip() or None}))+ return voice++[email protected]("/v1/voices")+async def list_voices():+ built_in = [{"id": name, "object": "audio.voice", "name": name, "type": "built_in"}+ for name in ENGINE.model.available_voices]+ saved = []+ for meta in ENGINE.voices_dir.glob("voice_*/voice.json"):+ voice = json.loads(meta.read_text())+ saved.append({"id": voice["id"], "object": "audio.voice", "name": voice["name"],+ "type": "custom", "created_at": voice["created_at"]})+ saved.sort(key=lambda v: v["created_at"], reverse=True)+ return {"object": "list", "data": built_in + saved}++[email protected]("/v1/audio/voices/{voice_id}")+async def delete_voice(voice_id: str):+ saved = load_saved_voice(voice_id)+ shutil.rmtree(ENGINE.voices_dir / voice_id, ignore_errors=True)+ ENGINE.pool.submit(ENGINE.forget, saved["reference"], saved["reference_transcript"])+ return {"id": voice_id, "object": "audio.voice", "deleted": True}+++def main():+ global ENGINE+ parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])+ parser.add_argument("--host", default="127.0.0.1")+ parser.add_argument("--port", type=int, default=8000)+ args = parser.parse_args()+ backend = os.environ.get("KITTENML_BACKEND", "vllm")+ if backend not in ("vllm", "torch"):+ raise SystemExit("KITTENML_BACKEND must be 'vllm' or 'torch'")+ vllm_options = json.loads(os.environ.get("KITTENML_VLLM_OPTIONS") or "{}")+ voices_dir = os.environ.get("KITTENML_VOICES_DIR",+ os.path.expanduser("~/.cache/kittenml/voices"))+ print(f"Loading KittenTTS 2 ({backend} backend)...", flush=True)+ ENGINE = Engine(backend, vllm_options, voices_dir)+ print(f"Ready: http://{args.host}:{args.port}", flush=True)+ uvicorn.run(app, host=args.host, port=args.port)+++if __name__ == "__main__":+ main()