Skip to content

SOTA-Class TTS at Compact Scale

Python

Add Windows PowerShell support for 0.1B ONNX runtime

chuarunyuan07e40f5d0b03
diff --git a/.gitignore b/.gitignore
index 6d4454a..f89a742 100644
--- a/.gitignore
+++ b/.gitignore
@@ -15,6 +15,8 @@ prepared_data/
/onnx_runtime/voices/
/onnx_runtime/outputs/
/onnx_runtime/.service.pid
+/onnx_runtime_0_1b_int8/.service.pid
+/onnx_runtime_0_1b_int8/voices/
# Model weights are distributed separately from the source repository.
/model/
diff --git a/onnx_runtime_0_1b_int8/README.md b/onnx_runtime_0_1b_int8/README.md
index ce8ae46..6f30272 100644
--- a/onnx_runtime_0_1b_int8/README.md
+++ b/onnx_runtime_0_1b_int8/README.md
@@ -33,6 +33,18 @@ bash setup.sh
python scripts/register_default_voice.py
```
+On Windows, use the native PowerShell entry points instead of `setup.sh`:
+
+```powershell
+py -3 -m pip install -U huggingface_hub
+hf download Audio8/audio8-TTS-0.1B-ONNX-INT8 --local-dir model
+.\setup.ps1
+.\.venv\Scripts\python.exe scripts\register_default_voice.py
+```
+
+If PowerShell blocks local scripts, run them with
+`powershell -ExecutionPolicy Bypass -File .\setup.ps1`.
+
The model directory must contain:
```text
@@ -65,6 +77,16 @@ bash run_infer.sh \
The command also writes `outputs/test.npy` with generated codes shaped
`[10, frames]`.
+The equivalent Windows command is:
+
+```powershell
+.\run_infer.ps1 `
+ --text "这是一个中文测试" `
+ --voice default `
+ --max-new-tokens 128 `
+ --output outputs\test.wav
+```
+
Set `ARKTTS_MODEL_DIR` or `ARKTTS_VOICES_DIR` to use another location. The
runtime always selects `CPUExecutionProvider`; GPU execution is outside the
scope of this export.
@@ -89,12 +111,26 @@ The service exposes `/api/tts`, `/api/tts/stream`, `/api/tts/cancel`,
`/api/voices`, `/api/voices/register`, `/api/registration/status`, and the
OpenAI-compatible `/v1/audio/speech`. Stop it with `bash stop_server.sh`.
+On Windows PowerShell, use the managed wrappers:
+
+```powershell
+.\start_server.ps1
+Invoke-WebRequest http://127.0.0.1:8024/api/health
+.\stop_server.ps1
+```
+
## Verification
```bash
"$PWD/.venv/bin/python" -m pytest -q tests
```
+On Windows:
+
+```powershell
+.\.venv\Scripts\python.exe -m pytest -q tests
+```
+
`test_contract.py` checks the exact input/output names and shapes when
`ARKTTS_MODEL_DIR` points at a downloaded model. Without model files it skips
the ONNX-dependent check; the pure prompt and state-shape tests still run.
diff --git a/onnx_runtime_0_1b_int8/README_zh.md b/onnx_runtime_0_1b_int8/README_zh.md
index 65f6158..19fb625 100644
--- a/onnx_runtime_0_1b_int8/README_zh.md
+++ b/onnx_runtime_0_1b_int8/README_zh.md
@@ -27,6 +27,18 @@ bash setup.sh
python scripts/register_default_voice.py
```
+Windows 用户请使用 PowerShell 脚本,不需要运行 `setup.sh`:
+
+```powershell
+py -3 -m pip install -U huggingface_hub
+hf download Audio8/audio8-TTS-0.1B-ONNX-INT8 --local-dir model
+.\setup.ps1
+.\.venv\Scripts\python.exe scripts\register_default_voice.py
+```
+
+如果 PowerShell 禁止执行本地脚本,可以运行:
+`powershell -ExecutionPolicy Bypass -File .\setup.ps1`。
+
模型文件放在本目录的 `model/` 下。`registration/` 下的 encoder 是可选的,
只在从音频注册新音色时需要。默认音色脚本只读取 `reference_codes.npy`。
@@ -43,6 +55,16 @@ bash run_infer.sh \
同时会生成 `[10, frames]` 形状的 `outputs/test.npy`。模型或音色放在其他位置时,
可设置 `ARKTTS_MODEL_DIR`、`ARKTTS_VOICES_DIR`。
+Windows PowerShell 推理命令:
+
+```powershell
+.\run_infer.ps1 `
+ --text "这是一个中文测试" `
+ --voice default `
+ --max-new-tokens 128 `
+ --output outputs\test.wav
+```
+
## HTTP 服务
服务同时提供与 0.6B ONNX Runtime 相同的本地网页界面。启动后打开
@@ -57,10 +79,24 @@ curl http://127.0.0.1:8024/api/health
服务提供 `/api/tts`、`/api/tts/stream`、`/api/tts/cancel`、音色查询和注册接口,
以及兼容 OpenAI 的 `/v1/audio/speech`。使用 `bash stop_server.sh` 停止服务。
+Windows PowerShell 使用:
+
+```powershell
+.\start_server.ps1
+Invoke-WebRequest http://127.0.0.1:8024/api/health
+.\stop_server.ps1
+```
+
## 测试
```bash
"$PWD/.venv/bin/python" -m pytest -q tests
```
+Windows:
+
+```powershell
+.\.venv\Scripts\python.exe -m pytest -q tests
+```
+
请在获得授权后进行音色克隆,并在适当场景披露合成音频。
diff --git a/onnx_runtime_0_1b_int8/arktts_runtime/service.py b/onnx_runtime_0_1b_int8/arktts_runtime/service.py
index e4d5288..7991f56 100644
--- a/onnx_runtime_0_1b_int8/arktts_runtime/service.py
+++ b/onnx_runtime_0_1b_int8/arktts_runtime/service.py
@@ -5,12 +5,17 @@ import gc
import io
import json
import os
-import resource
+import platform
import subprocess
import threading
import time
from pathlib import Path
+try:
+ import resource
+except ImportError: # Windows has no resource module.
+ resource = None # type: ignore[assignment]
+
import numpy as np
import soundfile as sf
from fastapi import FastAPI, File, Form, HTTPException, UploadFile
@@ -146,11 +151,12 @@ def _mac_current_memory_mb() -> float | None:
@app.get("/api/system")
def system_stats() -> dict:
current_mb, peak_mb = _linux_memory_mb()
- if current_mb is None and os.uname().sysname == "Darwin":
+ system = platform.system()
+ if current_mb is None and system == "Darwin":
current_mb = _mac_current_memory_mb()
- if peak_mb is None:
+ if peak_mb is None and resource is not None:
usage = float(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
- peak_mb = usage / (1024 * 1024) if os.uname().sysname == "Darwin" else usage / 1024
+ peak_mb = usage / (1024 * 1024) if system == "Darwin" else usage / 1024
return {
"memory": {"current_mb": current_mb, "peak_mb": peak_mb},
"uptime_seconds": time.monotonic() - service_started_at,
diff --git a/onnx_runtime_0_1b_int8/run_infer.ps1 b/onnx_runtime_0_1b_int8/run_infer.ps1
new file mode 100644
index 0000000..54f201a
--- /dev/null
+++ b/onnx_runtime_0_1b_int8/run_infer.ps1
@@ -0,0 +1,22 @@
+$ErrorActionPreference = "Stop"
+
+$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
+$venvPython = Join-Path $Root ".venv\Scripts\python.exe"
+if (-not (Test-Path -LiteralPath $venvPython -PathType Leaf)) {
+ throw "Virtual environment not found. Run setup.ps1 first."
+}
+
+$modelDir = if ([string]::IsNullOrWhiteSpace($env:ARKTTS_MODEL_DIR)) {
+ Join-Path $Root "model"
+} else { $env:ARKTTS_MODEL_DIR }
+$voicesDir = if ([string]::IsNullOrWhiteSpace($env:ARKTTS_VOICES_DIR)) {
+ Join-Path $Root "voices"
+} else { $env:ARKTTS_VOICES_DIR }
+
+Push-Location $Root
+try {
+ & $venvPython -m arktts_runtime.cli --model-dir $modelDir --voices-dir $voicesDir @args
+ exit $LASTEXITCODE
+} finally {
+ Pop-Location
+}
diff --git a/onnx_runtime_0_1b_int8/run_infer.sh b/onnx_runtime_0_1b_int8/run_infer.sh
index f652811..424a1b1 100755
--- a/onnx_runtime_0_1b_int8/run_infer.sh
+++ b/onnx_runtime_0_1b_int8/run_infer.sh
@@ -2,7 +2,15 @@
set -euo pipefail
ROOT="$(cd "$(dirname "$0")" && pwd)"
-exec "$ROOT/.venv/bin/python" -m arktts_runtime.cli \
+if [[ -x "$ROOT/.venv/bin/python" ]]; then
+ VENV_PYTHON="$ROOT/.venv/bin/python"
+elif [[ -x "$ROOT/.venv/Scripts/python.exe" ]]; then
+ VENV_PYTHON="$ROOT/.venv/Scripts/python.exe"
+else
+ echo "Virtual environment not found. Run setup.sh or setup.ps1 first." >&2
+ exit 1
+fi
+exec "$VENV_PYTHON" -m arktts_runtime.cli \
--model-dir "${ARKTTS_MODEL_DIR:-$ROOT/model}" \
--voices-dir "${ARKTTS_VOICES_DIR:-$ROOT/voices}" \
"$@"
diff --git a/onnx_runtime_0_1b_int8/run_server.ps1 b/onnx_runtime_0_1b_int8/run_server.ps1
new file mode 100644
index 0000000..a95f233
--- /dev/null
+++ b/onnx_runtime_0_1b_int8/run_server.ps1
@@ -0,0 +1,25 @@
+$ErrorActionPreference = "Stop"
+
+$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
+$venvPython = Join-Path $Root ".venv\Scripts\python.exe"
+if (-not (Test-Path -LiteralPath $venvPython -PathType Leaf)) {
+ throw "Virtual environment not found. Run setup.ps1 first."
+}
+
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_MODEL_DIR)) { $env:ARKTTS_MODEL_DIR = Join-Path $Root "model" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_VOICES_DIR)) { $env:ARKTTS_VOICES_DIR = Join-Path $Root "voices" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_REGISTRATION_DIR)) { $env:ARKTTS_REGISTRATION_DIR = Join-Path $env:ARKTTS_MODEL_DIR "registration" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_PRECISION)) { $env:ARKTTS_PRECISION = "int8" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_CODEC_PRECISION)) { $env:ARKTTS_CODEC_PRECISION = "fp16" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_THREADS)) { $env:ARKTTS_THREADS = "5" }
+
+$hostName = if ([string]::IsNullOrWhiteSpace($env:HOST)) { "127.0.0.1" } else { $env:HOST }
+$port = if ([string]::IsNullOrWhiteSpace($env:PORT)) { "8024" } else { $env:PORT }
+
+Push-Location $Root
+try {
+ & $venvPython -m uvicorn arktts_runtime.service:app --app-dir $Root --host $hostName --port $port @args
+ exit $LASTEXITCODE
+} finally {
+ Pop-Location
+}
diff --git a/onnx_runtime_0_1b_int8/run_server.sh b/onnx_runtime_0_1b_int8/run_server.sh
index 21ecde8..c4f85ad 100755
--- a/onnx_runtime_0_1b_int8/run_server.sh
+++ b/onnx_runtime_0_1b_int8/run_server.sh
@@ -2,6 +2,14 @@
set -euo pipefail
ROOT="$(cd "$(dirname "$0")" && pwd)"
+if [[ -x "$ROOT/.venv/bin/python" ]]; then
+ VENV_PYTHON="$ROOT/.venv/bin/python"
+elif [[ -x "$ROOT/.venv/Scripts/python.exe" ]]; then
+ VENV_PYTHON="$ROOT/.venv/Scripts/python.exe"
+else
+ echo "Virtual environment not found. Run setup.sh or setup.ps1 first." >&2
+ exit 1
+fi
export ARKTTS_MODEL_DIR="${ARKTTS_MODEL_DIR:-$ROOT/model}"
export ARKTTS_VOICES_DIR="${ARKTTS_VOICES_DIR:-$ROOT/voices}"
export ARKTTS_REGISTRATION_DIR="${ARKTTS_REGISTRATION_DIR:-$ARKTTS_MODEL_DIR/registration}"
@@ -9,7 +17,7 @@ export ARKTTS_PRECISION="${ARKTTS_PRECISION:-int8}"
export ARKTTS_CODEC_PRECISION="${ARKTTS_CODEC_PRECISION:-fp16}"
export ARKTTS_THREADS="${ARKTTS_THREADS:-5}"
-exec "$ROOT/.venv/bin/uvicorn" arktts_runtime.service:app \
+exec "$VENV_PYTHON" -m uvicorn arktts_runtime.service:app \
--app-dir "$ROOT" \
--host "${HOST:-127.0.0.1}" \
--port "${PORT:-8024}"
diff --git a/onnx_runtime_0_1b_int8/setup.ps1 b/onnx_runtime_0_1b_int8/setup.ps1
new file mode 100644
index 0000000..80c4fef
--- /dev/null
+++ b/onnx_runtime_0_1b_int8/setup.ps1
@@ -0,0 +1,48 @@
+$ErrorActionPreference = "Stop"
+
+$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
+$pythonExe = $env:PYTHON_BIN
+$pythonArgs = @()
+
+if ([string]::IsNullOrWhiteSpace($pythonExe)) {
+ $pyLauncher = Get-Command py -ErrorAction SilentlyContinue
+ if ($null -ne $pyLauncher) {
+ $pythonExe = $pyLauncher.Source
+ $pythonArgs = @("-3")
+ } else {
+ $pythonCommand = Get-Command python -ErrorAction SilentlyContinue
+ if ($null -eq $pythonCommand) {
+ $pythonCommand = Get-Command python3 -ErrorAction SilentlyContinue
+ }
+ if ($null -ne $pythonCommand) {
+ $pythonExe = $pythonCommand.Source
+ }
+ }
+}
+
+if ([string]::IsNullOrWhiteSpace($pythonExe)) {
+ throw "Python was not found. Install Python 3.10+ and add it to PATH."
+}
+
+& $pythonExe @pythonArgs -c "import sys; raise SystemExit(sys.version_info < (3, 10))"
+if ($LASTEXITCODE -ne 0) {
+ throw "Python 3.10 or newer is required."
+}
+
+$venv = Join-Path $Root ".venv"
+& $pythonExe @pythonArgs -m venv $venv
+if ($LASTEXITCODE -ne 0) {
+ throw "Failed to create the virtual environment at $venv."
+}
+
+$venvPython = Join-Path $venv "Scripts\python.exe"
+if (-not (Test-Path -LiteralPath $venvPython -PathType Leaf)) {
+ throw "The virtual environment was created, but $venvPython was not found."
+}
+
+& $venvPython -m pip install --upgrade pip
+if ($LASTEXITCODE -ne 0) { throw "Failed to upgrade pip." }
+& $venvPython -m pip install -r (Join-Path $Root "requirements.txt")
+if ($LASTEXITCODE -ne 0) { throw "Failed to install runtime requirements." }
+
+Write-Host "Environment ready: $venvPython"
diff --git a/onnx_runtime_0_1b_int8/setup.sh b/onnx_runtime_0_1b_int8/setup.sh
index 72cd5fb..be712e7 100755
--- a/onnx_runtime_0_1b_int8/setup.sh
+++ b/onnx_runtime_0_1b_int8/setup.sh
@@ -2,18 +2,33 @@
set -euo pipefail
ROOT="$(cd "$(dirname "$0")" && pwd)"
-PYTHON_BIN="${PYTHON_BIN:-python3}"
-
-if ! command -v "$PYTHON_BIN" >/dev/null 2>&1; then
- echo "Python was not found. Set PYTHON_BIN to a Python 3.10+ executable." >&2
+if [[ -n "${PYTHON_BIN:-}" ]]; then
+ PYTHON_CMD=("$PYTHON_BIN")
+elif command -v python3 >/dev/null 2>&1; then
+ PYTHON_CMD=(python3)
+elif command -v python >/dev/null 2>&1; then
+ PYTHON_CMD=(python)
+elif command -v py >/dev/null 2>&1; then
+ PYTHON_CMD=(py -3)
+else
+ echo "Python was not found. Install Python 3.10+ and ensure python, python3, or py is on PATH." >&2
exit 1
fi
-if ! "$PYTHON_BIN" -c 'import sys; raise SystemExit(sys.version_info < (3, 10))'; then
+
+if ! "${PYTHON_CMD[@]}" -c 'import sys; raise SystemExit(sys.version_info < (3, 10))'; then
echo "Python 3.10 or newer is required." >&2
exit 1
fi
-"$PYTHON_BIN" -m venv "$ROOT/.venv"
-"$ROOT/.venv/bin/python" -m pip install --upgrade pip
-"$ROOT/.venv/bin/python" -m pip install -r "$ROOT/requirements.txt"
-echo "Environment ready: $ROOT/.venv"
+"${PYTHON_CMD[@]}" -m venv "$ROOT/.venv"
+if [[ -x "$ROOT/.venv/bin/python" ]]; then
+ VENV_PYTHON="$ROOT/.venv/bin/python"
+elif [[ -x "$ROOT/.venv/Scripts/python.exe" ]]; then
+ VENV_PYTHON="$ROOT/.venv/Scripts/python.exe"
+else
+ echo "The virtual environment was created, but its Python executable was not found." >&2
+ exit 1
+fi
+"$VENV_PYTHON" -m pip install --upgrade pip
+"$VENV_PYTHON" -m pip install -r "$ROOT/requirements.txt"
+echo "Environment ready: $VENV_PYTHON"
diff --git a/onnx_runtime_0_1b_int8/start_server.ps1 b/onnx_runtime_0_1b_int8/start_server.ps1
new file mode 100644
index 0000000..3582e20
--- /dev/null
+++ b/onnx_runtime_0_1b_int8/start_server.ps1
@@ -0,0 +1,52 @@
+$ErrorActionPreference = "Stop"
+
+$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
+$venvPython = Join-Path $Root ".venv\Scripts\python.exe"
+if (-not (Test-Path -LiteralPath $venvPython -PathType Leaf)) {
+ throw "Virtual environment not found. Run setup.ps1 first."
+}
+
+$port = if ([string]::IsNullOrWhiteSpace($env:PORT)) { "8024" } else { $env:PORT }
+$pidFile = Join-Path $Root ".service.pid"
+if (Test-Path -LiteralPath $pidFile) {
+ $existingId = [int](Get-Content -LiteralPath $pidFile -Raw).Trim()
+ if (Get-Process -Id $existingId -ErrorAction SilentlyContinue) {
+ Write-Host "Service is already running with PID $existingId"
+ exit 0
+ }
+ Remove-Item -LiteralPath $pidFile -Force -ErrorAction SilentlyContinue
+}
+
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_MODEL_DIR)) { $env:ARKTTS_MODEL_DIR = Join-Path $Root "model" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_VOICES_DIR)) { $env:ARKTTS_VOICES_DIR = Join-Path $Root "voices" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_REGISTRATION_DIR)) { $env:ARKTTS_REGISTRATION_DIR = Join-Path $env:ARKTTS_MODEL_DIR "registration" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_PRECISION)) { $env:ARKTTS_PRECISION = "int8" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_CODEC_PRECISION)) { $env:ARKTTS_CODEC_PRECISION = "fp16" }
+if ([string]::IsNullOrWhiteSpace($env:ARKTTS_THREADS)) { $env:ARKTTS_THREADS = "5" }
+
+$logPath = Join-Path $Root "service.log"
+$errorPath = Join-Path $Root "service.error.log"
+$hostName = if ([string]::IsNullOrWhiteSpace($env:HOST)) { "127.0.0.1" } else { $env:HOST }
+$quotedRoot = '"' + $Root.Replace('"', '\"') + '"'
+$arguments = "-m uvicorn arktts_runtime.service:app --app-dir $quotedRoot --host $hostName --port $port"
+$process = Start-Process -FilePath $venvPython -ArgumentList $arguments -WorkingDirectory $Root `
+ -RedirectStandardOutput $logPath -RedirectStandardError $errorPath -PassThru
+$process.Id | Set-Content -LiteralPath $pidFile -Encoding ascii
+
+for ($attempt = 0; $attempt -lt 60; $attempt++) {
+ try {
+ $response = Invoke-WebRequest -UseBasicParsing -Uri "http://127.0.0.1:$port/api/health" -TimeoutSec 2
+ if ($response.StatusCode -eq 200) {
+ Write-Host "Audio8 0.1B TTS is ready: http://127.0.0.1:$port"
+ exit 0
+ }
+ } catch {
+ # The service can take time to load the ONNX sessions.
+ }
+ if ($process.HasExited) {
+ throw "Service exited during startup. See $logPath and $errorPath"
+ }
+ Start-Sleep -Seconds 1
+}
+
+throw "Service did not become ready in 60 seconds. See $logPath and $errorPath"
diff --git a/onnx_runtime_0_1b_int8/stop_server.ps1 b/onnx_runtime_0_1b_int8/stop_server.ps1
new file mode 100644
index 0000000..7a2c9a7
--- /dev/null
+++ b/onnx_runtime_0_1b_int8/stop_server.ps1
@@ -0,0 +1,17 @@
+$ErrorActionPreference = "Stop"
+
+$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
+$pidFile = Join-Path $Root ".service.pid"
+if (-not (Test-Path -LiteralPath $pidFile)) {
+ Write-Host "No managed Audio8 TTS service is running."
+ exit 0
+}
+
+$processId = [int](Get-Content -LiteralPath $pidFile -Raw).Trim()
+$process = Get-Process -Id $processId -ErrorAction SilentlyContinue
+if ($null -ne $process) {
+ Stop-Process -Id $processId
+ try { Wait-Process -Id $processId -Timeout 30 -ErrorAction Stop } catch { }
+}
+Remove-Item -LiteralPath $pidFile -Force -ErrorAction SilentlyContinue
+Write-Host "Audio8 TTS service stopped."
diff --git a/onnx_runtime_0_1b_int8/tests/test_service_platform.py b/onnx_runtime_0_1b_int8/tests/test_service_platform.py
new file mode 100644
index 0000000..c593801
--- /dev/null
+++ b/onnx_runtime_0_1b_int8/tests/test_service_platform.py
@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+
+def test_system_stats_does_not_require_unix_resource(monkeypatch) -> None:
+ import arktts_runtime.service as service
+
+ monkeypatch.setattr(service, "resource", None)
+ monkeypatch.setattr(service.platform, "system", lambda: "Windows")
+
+ result = service.system_stats()
+
+ assert result["memory"]["current_mb"] is None
+ assert result["memory"]["peak_mb"] is None
+ assert result["uptime_seconds"] >= 0