Skip to content

A platform for community discussion. Free, open, simple.

FEATURE: Per-user choice of quantized subtitles models in voice (#44434)

Rafael dos Santos Silvad3c3c6d45c56
diff --git a/Gemfile.lock b/Gemfile.lock
index 908b78ab3e6..01040dd1dcc 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -216,7 +216,7 @@ GEM
faker (~> 3.5.1)
literate_randomizer
discourse_math_bundle (1.0.1)
- discourse_voice_assets (0.1.0)
+ discourse_voice_assets (0.2.0)
drb (2.2.3)
dry-initializer (3.2.0)
ed25519 (1.4.0)
@@ -1047,7 +1047,7 @@ CHECKSUMS
discourse_ai-tokenizers (0.4.2) sha256=61c2f254582a3ae69255115b2b072904361003612b5b0f52aa64773817aae413
discourse_dev_assets (0.0.6) sha256=4dfe7946927aa3d61a4ba89b5aef39d6daaeb70e35d7125dc481806a5c0aea4e
discourse_math_bundle (1.0.1) sha256=c7d82fa65c9680cab2f77daeba32c1b89e500faccd2269e050ab27d5e1bc8337
- discourse_voice_assets (0.1.0) sha256=67bfbc3e9eaa1805e71ac3c5e2172d8beefdecbfa7b956ed5426e7e057c10cc2
+ discourse_voice_assets (0.2.0) sha256=dd94cca7787de0129fd3ec82c9bc8c178e5d49c7606b42f6dda167666b3e473e
drb (2.2.3) sha256=0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373
dry-initializer (3.2.0) sha256=37d59798f912dc0a1efe14a4db4a9306989007b302dcd5f25d0a2a20c166c4e3
ed25519 (1.4.0) sha256=16e97f5198689a154247169f3453ef4cfd3f7a47481fde0ae33206cdfdcac506
diff --git a/plugins/voice/README.md b/plugins/voice/README.md
index 390df9728da..399bd420892 100644
--- a/plugins/voice/README.md
+++ b/plugins/voice/README.md
@@ -108,17 +108,19 @@ Pre-built assets ship in the [discourse_voice_assets](https://github.com/discour
## Live Subtitles
-Opt-in, viewer-side captions: the user who enables subtitles transcribes the remote audio they already receive with [parakeet.js](https://github.com/ysdede/parakeet.js) (NVIDIA Parakeet TDT 0.6b v3, multilingual) — no audio leaves the browser and nothing is required from the other participants, on either transport.
+Opt-in, viewer-side captions: the user who enables subtitles transcribes the remote audio they already receive with [parakeet.js](https://github.com/ysdede/parakeet.js) running Moondream's Parakeet Ultra or Redux (fine-tunes of NVIDIA Parakeet TDT 0.6b v3, 25 languages) — no audio leaves the browser and nothing is required from the other participants, on either transport.
```
remote stream → Silero VAD (per participant) → utterance PCM → Worker (Parakeet, WebGPU) → caption overlay
```
-Each remote mic stream gets a [Silero VAD](https://github.com/ricky0123/vad) finding utterance boundaries; while a speaker keeps talking, the utterance-so-far is re-transcribed every ~1.5s as a provisional line that updates in place, and the speech-end pass finalizes it. Utterances are transcribed by one shared model in a Web Worker (fp32 encoder on WebGPU, int8 decoder on single-threaded WASM — fp16 encoders silently produce empty transcriptions on some GPU stacks, and multithreaded WASM would require COOP/COEP headers). Requires WebGPU; gated by the `voice_subtitles_enabled` site setting.
+Each remote mic stream gets a [Silero VAD](https://github.com/ricky0123/vad) finding utterance boundaries; while a speaker keeps talking, the utterance-so-far is re-transcribed every ~1.5s as a provisional line that updates in place, and the speech-end pass finalizes it. Utterances are transcribed by one shared model in a Web Worker (quantized encoder on WebGPU, int8 decoder on single-threaded WASM — multithreaded WASM would require COOP/COEP headers). Requires WebGPU; gated by the `voice_subtitles_enabled` site setting.
-The runtime bundles (worker, VAD, onnxruntime, ~41 MB) are pinned and shipped in the discourse_voice_assets gem (served here from `public/javascripts/<gem version>/stt/`, built by the gem's `scripts/build-stt-assets.sh`). The ~2.5 GB model weights are **not** committed: they download on first use from Discourse's HuggingFace repository (kept in a durable Cache API store), or from a self-hosted mirror configured via `voice_stt_model_base_url` — see [docs/subtitles-model-mirror.md](docs/subtitles-model-mirror.md) for what to mirror and how.
+Each user picks the model in their voice settings (stored per browser, like the noise-suppression mode; registry in `lib/voice/stt-models.js`): **Best quality** (Parakeet Ultra, 4-bit encoder, ~410 MB, the default) or **Smaller** (Parakeet Redux, ternary encoder stored as 2-bit, ~210 MB, less accurate in noise and in several languages). Switching reloads the worker on the new model, and the worker keeps only the current model in storage.
-An end-to-end smoke check (`scripts/smoke-stt-worker.mjs`, real model on WebGPU in headless Chromium) lives in the gem alongside the build script.
+The runtime bundles (worker, VAD, onnxruntime, ~30 MB) are pinned and shipped in the discourse_voice_assets gem (served here from `public/javascripts/<gem version>/stt/`, built by the gem's `scripts/build-stt-assets.sh`). The model weights are **not** committed: they download on first use from Discourse's HuggingFace repository ([Discourse/Discourse-STT](https://huggingface.co/Discourse/Discourse-STT), pinned to a commit; kept in a durable Cache API store), or from a self-hosted mirror configured via `voice_stt_model_base_url` — see [docs/subtitles-model-mirror.md](docs/subtitles-model-mirror.md) for what to mirror and how.
+
+An end-to-end smoke check (`scripts/smoke-stt-worker.mjs`, real model on WebGPU in Chromium) lives in the gem alongside the build script.
## Development
diff --git a/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.js b/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.js
index 04644e49634..ad4d413b560 100644
--- a/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.js
+++ b/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.js
@@ -521,6 +521,10 @@ export default class VoiceWebrtcService extends Service {
return this.#transcription.progress;
}
+ get sttModel() {
+ return this.#transcription.sttModel;
+ }
+
get captions() {
return this.#transcription.captions;
}
@@ -633,6 +637,11 @@ export default class VoiceWebrtcService extends Service {
this.#transcription.toggle();
}
+ @action
+ setSttModel(model) {
+ this.#transcription.setSttModel(model);
+ }
+
isTranscribingRoom(roomId) {
return this.#transcription.isTranscribingRoom(roomId);
}
diff --git a/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjs b/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjs
index 0e7f37c9e86..7b6179ab627 100644
--- a/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjs
+++ b/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjs
@@ -24,6 +24,7 @@ import {
engineForMode,
noiseSuppressionModeLabel,
} from "../../lib/voice/ns-engines";
+import { STT_MODELS, sttModelLabel } from "../../lib/voice/stt-models";
const METER_INTERVAL_MS = 50;
@@ -92,6 +93,13 @@ export default class VoiceVoiceSettingsModal extends Component {
}));
}
+ get sttModelOptions() {
+ return Object.keys(STT_MODELS).map((model) => ({
+ id: model,
+ name: sttModelLabel(model),
+ }));
+ }
+
get qualityOptions() {
return this.voiceWebrtc.allowedVoiceQualityTiers().map((tier) => ({
id: tier,
@@ -456,6 +464,22 @@ export default class VoiceVoiceSettingsModal extends Component {
{{i18n "voice.voice_settings.subtitles_hint"}}
</p>
</div>
+
+ <div class="voice-voice-settings__field">
+ <label class="voice-voice-settings__label">
+ {{i18n "voice.voice_settings.subtitles_model"}}
+ </label>
+ <ComboBox
+ class="voice-voice-settings__subtitles-model-select"
+ @content={{this.sttModelOptions}}
+ @onChange={{this.voiceWebrtc.setSttModel}}
+ @options={{hash none=false}}
+ @value={{this.voiceWebrtc.sttModel}}
+ />
+ <p class="voice-voice-settings__hint">
+ {{i18n "voice.voice_settings.subtitles_model_hint"}}
+ </p>
+ </div>
{{/if}}
{{#if this.showQuality}}
diff --git a/plugins/voice/assets/javascripts/discourse/lib/voice/stt-models.js b/plugins/voice/assets/javascripts/discourse/lib/voice/stt-models.js
new file mode 100644
index 00000000000..127fe5b4636
--- /dev/null
+++ b/plugins/voice/assets/javascripts/discourse/lib/voice/stt-models.js
@@ -0,0 +1,57 @@
+import { i18n } from "discourse-i18n";
+
+// The selectable speech-to-text models. Each is a folder holding the
+// worker's fixed file layout (encoder-model.onnx,
+// decoder_joint-model.int8.onnx, vocab.txt) under one model root: a site's
+// mirror (voice_stt_model_base_url) or Discourse's HuggingFace repository.
+// The default root is pinned to a commit because the worker caches by URL:
+// updated weights must arrive under a new URL to ever be downloaded.
+//
+// Order here is the order shown in the selector.
+export const STT_MODELS = {
+ ultra: { path: "ultra-q4" },
+ redux: { path: "redux-w2a8" },
+};
+
+export const DEFAULT_STT_MODEL = "ultra";
+
+const DEFAULT_MODEL_ROOT =
+ "https://huggingface.co/Discourse/Discourse-STT/resolve/0862913c8a5c08254d1082747c6443a06e062988";
+
+const STORAGE_KEY = "voice:stt-model";
+
+export function isSttModel(model) {
+ return Object.hasOwn(STT_MODELS, model);
+}
+
+export function sttModelUrl(model, modelRoot) {
+ const root = (modelRoot || DEFAULT_MODEL_ROOT).replace(/\/$/, "");
+ return `${root}/${STT_MODELS[model].path}`;
+}
+
+export function preferredSttModel() {
+ try {
+ const model = localStorage.getItem(STORAGE_KEY);
+ if (isSttModel(model)) {
+ return model;
+ }
+ } catch {
+ // fall through to the default
+ }
+ return DEFAULT_STT_MODEL;
+}
+
+export function setPreferredSttModel(model) {
+ if (!isSttModel(model)) {
+ return;
+ }
+ try {
+ localStorage.setItem(STORAGE_KEY, model);
+ } catch {
+ // ignore storage errors
+ }
+}
+
+export function sttModelLabel(model) {
+ return i18n(`voice.voice_settings.subtitles_models.${model}`);
+}
diff --git a/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.js b/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.js
index 3dabc99006f..6640de98a31 100644
--- a/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.js
+++ b/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.js
@@ -4,8 +4,8 @@ import { voiceAssetAppUrl, voiceAssetUrl } from "./voice-assets";
// Stable filenames in the discourse_voice_assets gem's stt/ directory.
const STT_WORKER_FILE = "subtitles-worker.js";
const VAD_BUNDLE_FILE = "vad.js";
-const ORT_WASM_JS_FILE = "ort/ort-wasm-simd-threaded.jsep.js";
-const ORT_WASM_BINARY_FILE = "ort/ort-wasm-simd-threaded.jsep.wasm";
+const ORT_WASM_JS_FILE = "ort/ort-wasm-simd-threaded.asyncify.js";
+const ORT_WASM_BINARY_FILE = "ort/ort-wasm-simd-threaded.asyncify.wasm";
const VAD_ASSET_DIR = "vad/";
// Live subtitles for room participants.
@@ -18,9 +18,10 @@ const VAD_ASSET_DIR = "vad/";
// Web Worker (WebGPU encoder), which serializes jobs across speakers.
//
// The runtime bundles are served from the plugin's public dir and only
-// fetched the first time subtitles are enabled; the ~2.5GB model weights
-// come from the `voice_stt_model_base_url` source (Discourse's
-// HuggingFace model repository by default), kept in a durable Cache API store.
+// fetched the first time subtitles are enabled; the model weights
+// (~200-400MB, see stt-models.js) come from the `voice_stt_model_base_url`
+// source (Discourse's HuggingFace model repository by default), kept in a
+// durable Cache API store.
const PREFERENCE_KEY = "voice:subtitles";
@@ -136,7 +137,19 @@ export default class SubtitlesManager {
}
setEnabled(enabled, { modelBaseUrl = null } = {}) {
+ const modelChanged = modelBaseUrl !== this.#modelBaseUrl;
this.#modelBaseUrl = modelBaseUrl;
+ if (modelChanged && this.#worker) {
+ // The worker holds one loaded model, so a different model needs a
+ // fresh one. Existing taps keep running and feed it once it's ready.
+ this.#terminateWorker();
+ if (enabled && this.#taps.size) {
+ this.#ensureWorker(modelBaseUrl);
+ } else {
+ this.#onLoadingChange();
+ }
+ }
+
if (this.#enabled === enabled) {
return;
}
@@ -306,7 +319,7 @@ export default class SubtitlesManager {
}
// Best effort, window-only API: asks the browser to exempt the origin's
- // storage (the multi-GB cached model) from eviction. Chromium decides
+ // storage (the cached model) from eviction. Chromium decides
// silently, but Firefox shows a permission prompt — so this must only
// run here, when transcription is actually starting in a room, never on
// plain page load.
@@ -340,7 +353,7 @@ export default class SubtitlesManager {
worker.postMessage({
type: "init",
config: {
- modelBaseUrl: modelBaseUrl || null,
+ modelBaseUrl,
ortWasmJsUrl: sttUrl(ORT_WASM_JS_FILE),
ortWasmBinaryUrl: sttUrl(ORT_WASM_BINARY_FILE),
},
diff --git a/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.js b/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.js
index 8ab259e4614..d97cac9e152 100644
--- a/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.js
+++ b/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.js
@@ -5,6 +5,12 @@ import Composer from "discourse/models/composer";
import Draft from "discourse/models/draft";
import { i18n } from "discourse-i18n";
import voiceLog from "discourse/plugins/voice/discourse/lib/voice/logger";
+import {
+ isSttModel,
+ preferredSttModel,
+ setPreferredSttModel,
+ sttModelUrl,
+} from "./stt-models";
import SubtitlesManager from "./subtitles";
import TranscriptDraftSync from "./transcript-draft-sync";
import { transcriptToMarkdown } from "./transcript-markdown";
@@ -20,6 +26,7 @@ export default class TranscriptionCoordinator {
@tracked progress = null;
@tracked captions = [];
@tracked revision = 0;
+ @tracked sttModel = preferredSttModel();
#subtitles;
#transcript;
@@ -119,7 +126,10 @@ export default class TranscriptionCoordinator {
}
get #modelBaseUrl() {
- return this.#siteSettings.voice_stt_model_base_url || null;
+ return sttModelUrl(
+ this.sttModel,
+ this.#siteSettings.voice_stt_model_base_url
+ );
}
get recording() {
@@ -158,6 +168,15 @@ export default class TranscriptionCoordinator {
this.#syncSttEngine();
}
+ setSttModel(model) {
+ if (!isSttModel(model) || model === this.sttModel) {
+ return;
+ }
+ this.sttModel = model;
+ setPreferredSttModel(model);
+ this.#syncSttEngine();
+ }
+
isTranscribingRoom(roomId) {
return this.recording && Number(this.roomId) === Number(roomId);
}
diff --git a/plugins/voice/config/locales/client.en.yml b/plugins/voice/config/locales/client.en.yml
index 898f0511273..50213e36725 100644
--- a/plugins/voice/config/locales/client.en.yml
+++ b/plugins/voice/config/locales/client.en.yml
@@ -172,7 +172,12 @@ en:
auto_gain_control: "Automatic gain control"
auto_gain_control_hint: "Keeps your volume steady by adjusting the microphone level automatically."
subtitles: "Show live subtitles"
- subtitles_hint: "Generates captions for what others say, on your device — nothing is sent anywhere. Requires a GPU-capable browser and downloads a ~2.5 GB model once."
+ subtitles_hint: "Generates captions for what others say, on your device — nothing is sent anywhere. Requires a GPU-capable browser and downloads the subtitles model once."
+ subtitles_model: "Subtitles model"
+ subtitles_model_hint: "Best quality is the most accurate. Smaller downloads about half as much, but makes more mistakes in noisy rooms and in some languages."
+ subtitles_models:
+ ultra: "Best quality (~410 MB)"
+ redux: "Smaller (~210 MB)"
subtitles_failed: "Subtitles couldn't be started, so they were turned off."
test_output: "Test"
choose_output: "Choose output device"
diff --git a/plugins/voice/config/locales/server.en.yml b/plugins/voice/config/locales/server.en.yml
index d3df172f6f5..ced29985c6a 100644
--- a/plugins/voice/config/locales/server.en.yml
+++ b/plugins/voice/config/locales/server.en.yml
@@ -22,8 +22,8 @@ en:
voice_screen_share_allowed_groups: "Groups whose members can share their screen in voice rooms. Leave empty to turn screen sharing off site-wide. Rooms can individually opt out of all media via their room settings, and stage room listeners can never publish."
voice_video_max_publishers: "Maximum number of participants who can share video or their screen at the same time in a room. On peer-to-peer rooms, higher values increase CPU and bandwidth load for everyone watching; on rooms using a LiveKit media server, they mainly increase load on that server."
voice_video_background_blur_enabled: "Allow participants to blur their camera background. Segmentation runs on the publisher's device via MediaPipe, which increases CPU/GPU load while their camera is on."
- voice_subtitles_enabled: "Allow participants to turn on live subtitles for other people's speech. Transcription runs entirely on the viewer's device via an NVIDIA Parakeet speech-to-text model (~2.5 GB, downloaded on first use, WebGPU required); no audio leaves the browser."
- voice_stt_model_base_url: "Optional base URL for a self-hosted mirror of the subtitles model. When blank, the model files are downloaded on first use from Discourse's HuggingFace repository (Discourse/parakeet-tdt-0.6b-v3-onnx). A mirror must serve the same four files under their original names, reachable by browsers with CORS allowed — see docs/subtitles-model-mirror.md in the plugin."
+ voice_subtitles_enabled: "Allow participants to turn on live subtitles for other people's speech. Transcription runs entirely on the viewer's device via an NVIDIA Parakeet speech-to-text model (less than 0.5 GB, downloaded on first use, WebGPU required); no audio leaves the browser."
+ voice_stt_model_base_url: "Optional base URL for a self-hosted mirror of the subtitles models. When blank, the model files are downloaded on first use from Discourse's HuggingFace repository (Discourse/Discourse-STT). A mirror must keep that repository's folder layout and file names, reachable by browsers with CORS allowed — see docs/subtitles-model-mirror.md in the plugin."
voice_max_voice_quality: "Highest microphone audio quality participants may choose in their voice settings. Standard is the browser's speech default (~32 kbps); high (~64 kbps) and maximum (~128 kbps) improve fidelity for good microphones at the cost of bandwidth."
voice_max_camera_quality: "Highest camera quality participants may choose in their video settings. Standard captures at 720p; high captures at 1080p; maximum additionally stops automatic quality reduction as viewer count grows on peer-to-peer rooms, which multiplies the sender's upload bandwidth and CPU use."
voice_max_screen_share_quality: "Highest screen share quality participants may choose in their video settings. Standard targets 15 fps at a modest bitrate; high allows 30 fps at a higher bitrate; maximum allows 60 fps at a very high bitrate ceiling, letting fast connections share high-resolution screens at full quality."
diff --git a/plugins/voice/docs/subtitles-fp16-encoder.md b/plugins/voice/docs/subtitles-fp16-encoder.md
deleted file mode 100644
index 2c7dd45c463..00000000000
--- a/plugins/voice/docs/subtitles-fp16-encoder.md
+++ /dev/null
@@ -1,166 +0,0 @@
-# Subtitles: fp16 encoder
-
-Status (2026-08-26): **auto-selected and validated.** The worker picks fp16
-when the adapter exposes `shader-f16` and fp32 otherwise (Chromium on
-Linux). Root cause of the historical empty-string failure is diagnosed
-below. The fp16 path is validated end-to-end on real f16 hardware via
-Firefox on Linux (`STT_BROWSER=firefox` in the smoke harness), which —
-unlike Chromium — exposes `shader-f16` there.
-
-## Why we want it
-
-The Parakeet encoder runs on WebGPU in fp32 today:
-
-- Weights: `encoder-model.onnx.data` is ~2.4 GB in fp32; an fp16 encoder is
- ~1.2 GB — half the download, disk cache, and resident VRAM.
-- Activations: onnxruntime-web's WebGPU execution provider keeps every freed
- GPU buffer in a size-keyed free list for the lifetime of the session (no
- eviction, no return to the driver). Since utterances arrive in many
- distinct lengths, the pool converges to the union of all activation-buffer
- sizes ever seen — observed growing from ~3.2 GB to ~5.5 GB over an hour of
- live captioning. fp16 would halve every one of those buffers, on top of
- halving the weights.
-
-On low-VRAM devices (e.g. 8 GB shared with the compositor and video decode)
-the fp32 footprint risks driver paging (massive slowdown) and eventually
-WebGPU device loss.
-
-## Symptom
-
-With an fp16 encoder, transcription **silently returns empty strings** on
-some machines — no error, no rejected promise, just blank output. The
-upstream demo (<https://ysdede.github.io/keet/>) shows the same failure
-here. This is noted in `src/stt-worker/worker.js` next to the
-`encoderQuant` default.
-
-## Root cause (diagnosed 2026-08-25)
-
-Three findings, established empirically on a Linux machine that reproduces
-the failure (RTX 5090, NVIDIA 610.57.04, Chromium 151):
-
-1. **Chromium on Linux does not expose the WebGPU `shader-f16` feature at
- all** — on any GPU. The Vulkan driver advertises `shaderFloat16` and
- 16-bit storage, but the adapter's feature set lacks `shader-f16`, and no
- flag (`--enable-dawn-features`, `--enable-blink-features`,
- `--enable-webgpu-developer-features`) turns it on. Per the Chromium
- intent-to-ship for WebGPU f16, the feature shipped on macOS/ChromeOS
- first and on Windows behind the DXC migration; Linux never got it.
- "Works on some machines, fails on others" is **feature availability by
- platform**, not driver numerics. (Firefox's WebGPU on the same
- machine/driver _does_ expose `shader-f16` — the gap is Chromium's, not
- Vulkan's.)
-2. **onnxruntime-web runs fp16 models anyway when the feature is missing.**
- The JSEP WebGPU EP requests `shader-f16` on the device and prepends
- `enable f16;` to WGSL _only if the adapter has it_, but never refuses an
- fp16 model when it doesn't — execution proceeds and produces garbage
- encodings, the decoder emits nothing, and no error surfaces. (Upstream
- bug worth filing: this should be a hard error or a fallback.)
-3. **The fp16 exports themselves are numerically fine.** On the CPU EP,
- both `grikdotnet/parakeet-tdt-0.6b-fp16` (full fp16, fp32 IO) and our
- own mixed-precision exports match the fp32 encoder to cosine similarity
- ≥ 0.999998 with zero NaN/Inf.
-
-The earlier overflow hypothesis is **disproved** for this model: scanning
-all float intermediates of the fp32 encoder over speech fixtures
-(`scripts/scan_overflow.py`) shows a global max |activation| of exactly
-10000.0 (the attention-mask fill constant) — far inside fp16's 65504 range.
-The transformers.js Whisper issue we cited
-(<https://github.com/huggingface/transformers.js/issues/1590>) is a
-different failure class.
-
-Community fp16 exports, for the record: `grikdotnet/parakeet-tdt-0.6b-fp16`
-(numerically clean, CUDA-validated only) and
-`Olicorne/parakeet-tdt-0.6b-v3-optimized-onnx` (fp16 files were a naive
-cast, withdrawn by the author 2026-08-23; repo remains interesting for its
-sharded fp32 encoder). Neither fails because of how it was exported — any
-fp16 export fails wherever `shader-f16` is unavailable.
-
-## Current behavior
-
-The worker (`src/stt-worker/worker.js`) resolves the encoder precision at
-init: an explicit `encoderQuant` of `"fp16"`/`"fp32"` is honored (the smoke
-harness forces it via `STT_ENCODER_QUANT`), anything else auto-selects via
-`adapter.features.has("shader-f16")`. fp16 loads the self-contained
-`encoder-model.fp16.onnx` from our model repo (fp32 IO, so parakeet.js's
-float32 feeds work unchanged); fp32 keeps the sidecar pair. Today that
-means fp16 for macOS/ChromeOS/newer-Windows Chrome, fp32 for all of Linux.
-
-Verified both ways on the same machine (RTX 5090):
-
-- Chromium (no `shader-f16`): auto picks fp32 and transcribes; forced fp16
- reproduces the empty-string failure, as expected.
-- Firefox (`shader-f16` exposed): auto picks fp16 and the full smoke
- harness passes — correct transcription and VAD utterances with the
- shipped `encoder-model.fp16.onnx`.
-
-## Still open
-
-1. Consider an init warm-up canary (a fixture must transcribe to non-empty
- text, else re-init as fp32) as a belt-and-braces fallback for genuinely
- broken f16 drivers.
-2. Watch for Chromium enabling `shader-f16` on Linux, and consider filing
- the ORT-web silent-execution bug upstream.
-3. The f16 kernels are only exercised on Firefox/Vulkan so far; Chromium's
- Metal/D3D backends (where it will actually run fp16 in production) run
- the same ORT WGSL but haven't been checked by us directly.
-
-Note: int8/fp8 are not alternatives for the encoder — WGSL has no 8-bit
-types (only packed DP4a-style dot products), ORT-web falls back off the
-WebGPU EP for quantized encoder ops (parakeet.js upgrades int8 encoder
-requests to fp32 on WebGPU backends), and fp8 does not exist in WebGPU at
-all. The int8 decoder already runs on WASM, unaffected.
-
-## Reproducing the diagnosis
-
-Checking any adapter takes one line in a devtools console:
-
-```js
-(await navigator.gpu.requestAdapter()).features.has("shader-f16");
-```
-
-The numeric findings came from running candidate encoders on
-onnxruntime's CPU EP against the fp32 reference (same mel input, compare
-outputs for NaN/Inf and cosine similarity), and from running the fp32
-encoder with all intermediates exposed as graph outputs to scan activation
-ranges. Mixed-precision exports were produced with onnxconverter-common's
-`convert_float_to_float16` (`keep_io_types=True`); none of that tooling is
-needed going forward, since the shipped fp16 file already exists in the
-model repo.
-
-The smoke harness (`scripts/smoke-stt-worker.mjs` in the
-discourse_voice_assets gem repository) now logs the adapter and
-launches Chromium with `Vulkan,VulkanFromANGLE,DefaultANGLEVulkan` —
-without the FromANGLE pair, headless Chromium silently falls back to
-SwiftShader (CPU rasterizer, no `shader-f16`, ~1000× slower encode), which
-is both a useless benchmark and an accidental repro of the no-f16 failure.
-
-## How to test
-
-The smoke harness exercises the real worker in headless Chromium; point it
-at a directory containing candidate files (the worker fetches
-`encoder-model.onnx`, `encoder-model.onnx.data` — a zero-byte file works
-for self-contained encoders — `decoder_joint-model.int8.onnx`, `vocab.txt`):
-
-```bash
-flite -t "the quick brown fox jumps over the lazy dog" /tmp/fix.wav
-STT_MODEL_DIR=/path/to/model-dir node scripts/smoke-stt-worker.mjs /tmp/fix.wav
-```
-
-Check the `adapter:` line in the output: on an adapter without
-`shader-f16`, an fp16-encoder FAIL only reproduces the availability bug and
-says nothing about the export's numerics. Use `STT_SMOKE_PORT` to give each
-model dir its own origin, since downloads are cached per-origin in the
-persistent profile.
-
-To exercise the fp16 path on Linux, run the harness in Firefox — the only
-browser exposing `shader-f16` here (WebGPU is enabled via prefs in a
-separate persistent profile):
-
-```bash
-STT_BROWSER=firefox STT_MODEL_DIR=/path/to/model-dir \
- node scripts/smoke-stt-worker.mjs /tmp/fix.wav
-```
-
-The model dir needs `encoder-model.fp16.onnx` (self-contained; a zero-byte
-`encoder-model.onnx.data` still satisfies the fp32 branch if you force it)
-— auto-select picks fp16 there and the whole chain should PASS.
diff --git a/plugins/voice/docs/subtitles-model-mirror.md b/plugins/voice/docs/subtitles-model-mirror.md
index c26ab6fa477..341078cd0cf 100644
--- a/plugins/voice/docs/subtitles-model-mirror.md
+++ b/plugins/voice/docs/subtitles-model-mirror.md
@@ -1,12 +1,13 @@
-# Self-hosting the subtitles model (mirror)
-
-By default, live subtitles download the speech-to-text model on first use
-from Discourse's HuggingFace repository
-([Discourse/parakeet-tdt-0.6b-v3-onnx](https://huggingface.co/Discourse/parakeet-tdt-0.6b-v3-onnx),
-a clone of the upstream `ysdede/parakeet-tdt-0.6b-v3-onnx` export). Sites
-that can't (or don't want to) let browsers reach `huggingface.co` —
-airgapped deployments, enterprise networks, or anyone who prefers to
-control the bytes — can serve the model themselves and point the
+# Self-hosting the subtitles models (mirror)
+
+By default, live subtitles download their speech-to-text model on first use
+from Discourse's HuggingFace repository,
+[Discourse/Discourse-STT](https://huggingface.co/Discourse/Discourse-STT).
+The plugin pins that repository to a specific commit.
+
+Some sites can't let browsers reach `huggingface.co`, or prefer not to:
+airgapped deployments, enterprise networks, or anyone who wants to control
+the bytes. They can serve the models themselves and point the
`voice_stt_model_base_url` site setting at their own mirror.
Only the **model weights** are affected. The runtime bundles (worker, VAD,
@@ -15,79 +16,102 @@ need no extra hosting.
## What to mirror
-Four files from the
-[ysdede/parakeet-tdt-0.6b-v3-onnx](https://huggingface.co/ysdede/parakeet-tdt-0.6b-v3-onnx)
-repository (models are CC-BY-4.0; keep the repo's attribution requirements in
-mind), **keeping their exact filenames**, in a single flat directory:
+The repository has one folder per model. Each user picks a model in their
+voice settings, so mirror every folder. **Keep the folder names and file
+names exactly as they are.**
-| File | Size | Purpose |
+| Folder | Model | Size |
| --- | --- | --- |
-| `encoder-model.onnx` | ~42 MB | fp32 encoder graph (runs on WebGPU) |
-| `encoder-model.onnx.data` | ~2.4 GB | encoder weights (external data referenced by the graph — the filename must not change) |
-| `decoder_joint-model.int8.onnx` | ~18 MB | int8 decoder/joint (runs on WASM) |
-| `vocab.txt` | ~100 KB | tokenizer vocabulary |
+| `ultra-q4/` | Parakeet Ultra, 4-bit encoder (default, "Best quality") | ~410 MB |
+| `redux-w2a8/` | Parakeet Redux, 2-bit encoder ("Smaller") | ~210 MB |
+
+Every folder holds the same three files:
+
+| File | Purpose |
+| --- | --- |
+| `encoder-model.onnx` | quantized encoder, a single file (runs on WebGPU) |
+| `decoder_joint-model.int8.onnx` | int8 decoder/joint (runs on WASM) |
+| `vocab.txt` | tokenizer vocabulary |
+
+To download them, using the commit the plugin pins (see `DEFAULT_MODEL_ROOT`
+in `assets/javascripts/discourse/lib/voice/stt-models.js`):
```bash
-mkdir parakeet-tdt-0.6b-v3
-cd parakeet-tdt-0.6b-v3
-for f in encoder-model.onnx encoder-model.onnx.data \
- decoder_joint-model.int8.onnx vocab.txt; do
- curl -LO "https://huggingface.co/ysdede/parakeet-tdt-0.6b-v3-onnx/resolve/main/$f"
+REV=0862913c8a5c08254d1082747c6443a06e062988
+for model in ultra-q4 redux-w2a8; do
+ mkdir -p "discourse-stt/$model"
+ for f in encoder-model.onnx decoder_joint-model.int8.onnx vocab.txt; do
+ curl -L -o "discourse-stt/$model/$f" \
+ "https://huggingface.co/Discourse/Discourse-STT/resolve/$REV/$model/$f"
+ done
done
```
-No subdirectory structure is required or supported: the plugin requests
-`<base_url>/<filename>` for each of the four files, nothing else.
+The models are CC-BY-4.0. The repository's README lists the attribution and
+the source of every file.
+
+The plugin requests `<base_url>/<folder>/<file>` and nothing else.
## Hosting requirements
-The files are fetched by the **browser** (from a Web Worker on the forum's
-origin), not by the Discourse server, so the mirror must be reachable by your
-users and must speak CORS if it lives on a different origin:
+The files are fetched by the **browser**, from a Web Worker on the forum's
+origin, not by the Discourse server. So the mirror must be reachable by your
+users, and must speak CORS if it lives on a different origin:
-- `Access-Control-Allow-Origin: <forum origin>` (or `*`) on all four files.
+- `Access-Control-Allow-Origin: <forum origin>` (or `*`) on every file.
- HTTPS, if the forum is served over HTTPS (mixed content is blocked).
-- Correct `Content-Length` (any static file server does this) — it drives
- the download progress shown in the caption overlay, and the ~2.4 GB file
- arrives as one plain GET; resumable/range support is not needed.
-- Long-lived cache headers are still recommended (the files are immutable),
- though the browser keeps a durable Cache API copy after the first
- download, so mirrors are only re-hit when that storage is evicted.
+- A correct `Content-Length`, which any static file server sends. It drives
+ the download progress shown in the caption overlay. Range requests are
+ not needed.
+- Long-lived cache headers are recommended, because the files never change.
Example nginx location:
```nginx
-location /models/parakeet-tdt-0.6b-v3/ {
+location /models/discourse-stt/ {
root /var/www;
add_header Access-Control-Allow-Origin "https://forum.example.com";
add_header Cache-Control "public, max-age=31536000, immutable";
}
```
-Then set the site setting:
+Then set the site setting (the trailing slash is optional):
```
-voice_stt_model_base_url = https://cdn.example.com/models/parakeet-tdt-0.6b-v3
+voice_stt_model_base_url = https://cdn.example.com/models/discourse-stt
```
-(Trailing slash optional.)
+## Updating a mirror
+
+Browsers keep a durable copy of the model they use and look it up by URL.
+If you replace a file at the same URL, users who already have the old copy
+keep it.
-A custom mirror behaves exactly like the default source: streamed download
-progress in the caption overlay and a durable Cache API copy on the user's
-device.
+When the plugin moves to a new model revision, publish the new files under a
+new directory and point the setting there. For example, put the revision in
+the path: `.../discourse-stt/<rev>/`.
+
+Each browser keeps only one model. Switching models, or changing the mirror
+URL, frees the previous copy.
## Verifying a mirror
-The end-to-end smoke harness (in the discourse_voice_assets gem repository,
-alongside the shipped worker bundle) can run against a local copy of the
-mirror directory:
+The end-to-end smoke harness lives in the discourse_voice_assets gem
+repository, alongside the shipped worker bundle. It can run against a local
+copy of any one model folder:
```bash
flite -t "the quick brown fox jumps over the lazy dog" /tmp/fix.wav
-STT_MODEL_DIR=/path/to/parakeet-tdt-0.6b-v3 \
+STT_MODEL_DIR=/path/to/discourse-stt/ultra-q4 \
+ node scripts/smoke-stt-worker.mjs /tmp/fix.wav
+```
+
+Or point it at the mirror itself:
+
+```bash
+STT_MODEL_BASE_URL=https://cdn.example.com/models/discourse-stt/ultra-q4 \
node scripts/smoke-stt-worker.mjs /tmp/fix.wav
```
-It loads the shipped worker in headless Chromium (WebGPU required), serves
-the model files from the given directory exactly as a mirror would, and
-fails unless the fixture transcribes.
+The harness loads the shipped worker in Chromium (WebGPU required) and fails
+unless the fixture transcribes.
diff --git a/plugins/voice/spec/integrity/voice_assets_spec.rb b/plugins/voice/spec/integrity/voice_assets_spec.rb
index 6f91d897851..af9735143bf 100644
--- a/plugins/voice/spec/integrity/voice_assets_spec.rb
+++ b/plugins/voice/spec/integrity/voice_assets_spec.rb
@@ -19,8 +19,8 @@ RSpec.describe "discourse_voice_assets integrity" do
rnnoise/rnnoise.wasm
stt/subtitles-worker.js
stt/vad.js
- stt/ort/ort-wasm-simd-threaded.jsep.js
- stt/ort/ort-wasm-simd-threaded.jsep.wasm
+ stt/ort/ort-wasm-simd-threaded.asyncify.js
+ stt/ort/ort-wasm-simd-threaded.asyncify.wasm
stt/vad/
livekit/livekit-client.js
mediapipe/vision_bundle.js
diff --git a/plugins/voice/spec/system/voice_subtitles_spec.rb b/plugins/voice/spec/system/voice_subtitles_spec.rb
index 6feb048f6b4..976eeae72e2 100644
--- a/plugins/voice/spec/system/voice_subtitles_spec.rb
+++ b/plugins/voice/spec/system/voice_subtitles_spec.rb
@@ -69,4 +69,20 @@ describe "Voice subtitles" do
find(".modal-close").click
expect(page).to have_css(".voice-captions", visible: :all)
end
+
+ it "lets the user pick the smaller subtitles model" do
+ SiteSetting.voice_subtitles_enabled = true
+ stub_webgpu
+ join_room
+ open_voice_settings
+
+ select = PageObjects::Components::SelectKit.new(".voice-voice-settings__subtitles-model-select")
+ expect(select).to have_selected_name(I18n.t("js.voice.voice_settings.subtitles_models.ultra"))
+
+ select.expand
+ select.select_row_by_value("redux")
+
+ expect(select).to have_selected_name(I18n.t("js.voice.voice_settings.subtitles_models.redux"))
+ expect(page.evaluate_script("localStorage.getItem('voice:stt-model')")).to eq("redux")
+ end
end
diff --git a/plugins/voice/test/javascripts/unit/lib/stt-models-test.js b/plugins/voice/test/javascripts/unit/lib/stt-models-test.js
new file mode 100644
index 00000000000..62b36747c0b
--- /dev/null
+++ b/plugins/voice/test/javascripts/unit/lib/stt-models-test.js
@@ -0,0 +1,59 @@
+import { setupTest } from "ember-qunit";
+import { module, test } from "qunit";
+import {
+ DEFAULT_STT_MODEL,
+ preferredSttModel,
+ setPreferredSttModel,
+ STT_MODELS,
+ sttModelUrl,
+} from "discourse/plugins/voice/discourse/lib/voice/stt-models";
+
+module("Voice | Unit | Lib | stt-models", function (hooks) {
+ setupTest(hooks);
+
+ hooks.beforeEach(function () {
+ localStorage.removeItem("voice:stt-model");
+ });
+
+ hooks.afterEach(function () {
+ localStorage.removeItem("voice:stt-model");
+ });
+
+ test("defaults to the best-quality model", function (assert) {
+ assert.strictEqual(DEFAULT_STT_MODEL, "ultra");
+ assert.strictEqual(preferredSttModel(), "ultra");
+ });
+
+ test("remembers a chosen model and ignores unknown ones", function (assert) {
+ setPreferredSttModel("redux");
+ assert.strictEqual(preferredSttModel(), "redux");
+
+ setPreferredSttModel("v3");
+ assert.strictEqual(
+ preferredSttModel(),
+ "redux",
+ "unknown ids aren't stored"
+ );
+
+ localStorage.setItem("voice:stt-model", "v3");
+ assert.strictEqual(
+ preferredSttModel(),
+ "ultra",
+ "a stale stored id falls back to the default"
+ );
+ });
+
+ test("model URLs are folders under the pinned repository or a mirror", function (assert) {
+ assert.true(
+ /^https:\/\/huggingface\.co\/Discourse\/Discourse-STT\/resolve\/[0-9a-f]{40}\/ultra-q4$/.test(
+ sttModelUrl("ultra")
+ ),
+ "the default root is pinned to a commit"
+ );
+ assert.strictEqual(
+ sttModelUrl("redux", "https://cdn.example.com/stt/"),
+ "https://cdn.example.com/stt/redux-w2a8"
+ );
+ assert.deepEqual(Object.keys(STT_MODELS), ["ultra", "redux"]);
+ });
+});
diff --git a/plugins/voice/test/javascripts/unit/lib/subtitles-test.js b/plugins/voice/test/javascripts/unit/lib/subtitles-test.js
index 26bec43a9b4..5783d7fad21 100644
--- a/plugins/voice/test/javascripts/unit/lib/subtitles-test.js
+++ b/plugins/voice/test/javascripts/unit/lib/subtitles-test.js
@@ -351,4 +351,54 @@ module("Voice | Unit | Lib | subtitles", function (hooks) {
assert.strictEqual(job.userId, 9);
assert.strictEqual(job.pcm.byteLength, 8000 * 4);
});
+
+ test("the worker is initialized with the chosen model's URL", async function (assert) {
+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/ultra" });
+ await this.manager.attach(1, 42, createFakeStream("s1"));
+
+ const init = FakeWorker.instances[0].messages.find(
+ (message) => message.type === "init"
+ );
+ assert.strictEqual(init.config.modelBaseUrl, "https://models/ultra");
+ assert.true(
+ init.config.ortWasmBinaryUrl.endsWith(
+ "stt/ort/ort-wasm-simd-threaded.asyncify.wasm"
+ )
+ );
+ });
+
+ test("switching models replaces the worker and keeps the taps", async function (assert) {
+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/ultra" });
+ await this.manager.attach(1, 42, createFakeStream("s1"));
+ await new Promise((resolve) => setTimeout(resolve, 0));
+
+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/redux" });
+
+ assert.true(FakeWorker.instances[0].terminated, "old worker terminated");
+ assert.strictEqual(FakeWorker.instances.length, 2, "new worker started");
+ assert.strictEqual(
+ FakeWorker.instances[1].messages[0].config.modelBaseUrl,
+ "https://models/redux"
+ );
+ assert.false(FakeVad.instances[0].destroyed, "the tap survives");
+
+ await new Promise((resolve) => setTimeout(resolve, 0));
+ FakeVad.instances[0].speak();
+ await new Promise((resolve) => setTimeout(resolve, 10));
+
+ assert.strictEqual(FakeWorker.instances[1].jobs.length, 1);
+ assert.strictEqual(this.captions.length, 1, "captions keep flowing");
+ });
+
+ test("switching models without taps defers the new worker", async function (assert) {
+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/ultra" });
+ await this.manager.attach(1, 42, createFakeStream("s1"));
+ this.manager.detachRoom(1);
+
+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/redux" });
+
+ assert.true(FakeWorker.instances[0].terminated);
+ assert.strictEqual(FakeWorker.instances.length, 1, "nothing to transcribe");
+ assert.false(this.manager.loading);
+ });
});