A platform for community discussion. Free, open, simple.
FEATURE: Per-user choice of quantized subtitles models in voice (#44434)
Rafael dos Santos Silvad3c3c6d45c56
diff --git a/Gemfile.lock b/Gemfile.lockindex 908b78ab3e6..01040dd1dcc 100644--- a/Gemfile.lock+++ b/Gemfile.lock@@ -216,7 +216,7 @@ GEMfaker (~> 3.5.1)literate_randomizerdiscourse_math_bundle (1.0.1)- discourse_voice_assets (0.1.0)+ discourse_voice_assets (0.2.0)drb (2.2.3)dry-initializer (3.2.0)ed25519 (1.4.0)@@ -1047,7 +1047,7 @@ CHECKSUMSdiscourse_ai-tokenizers (0.4.2) sha256=61c2f254582a3ae69255115b2b072904361003612b5b0f52aa64773817aae413discourse_dev_assets (0.0.6) sha256=4dfe7946927aa3d61a4ba89b5aef39d6daaeb70e35d7125dc481806a5c0aea4ediscourse_math_bundle (1.0.1) sha256=c7d82fa65c9680cab2f77daeba32c1b89e500faccd2269e050ab27d5e1bc8337- discourse_voice_assets (0.1.0) sha256=67bfbc3e9eaa1805e71ac3c5e2172d8beefdecbfa7b956ed5426e7e057c10cc2+ discourse_voice_assets (0.2.0) sha256=dd94cca7787de0129fd3ec82c9bc8c178e5d49c7606b42f6dda167666b3e473edrb (2.2.3) sha256=0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373dry-initializer (3.2.0) sha256=37d59798f912dc0a1efe14a4db4a9306989007b302dcd5f25d0a2a20c166c4e3ed25519 (1.4.0) sha256=16e97f5198689a154247169f3453ef4cfd3f7a47481fde0ae33206cdfdcac506diff --git a/plugins/voice/README.md b/plugins/voice/README.mdindex 390df9728da..399bd420892 100644--- a/plugins/voice/README.md+++ b/plugins/voice/README.md@@ -108,17 +108,19 @@ Pre-built assets ship in the [discourse_voice_assets](https://github.com/discour## Live Subtitles-Opt-in, viewer-side captions: the user who enables subtitles transcribes the remote audio they already receive with [parakeet.js](https://github.com/ysdede/parakeet.js) (NVIDIA Parakeet TDT 0.6b v3, multilingual) — no audio leaves the browser and nothing is required from the other participants, on either transport.+Opt-in, viewer-side captions: the user who enables subtitles transcribes the remote audio they already receive with [parakeet.js](https://github.com/ysdede/parakeet.js) running Moondream's Parakeet Ultra or Redux (fine-tunes of NVIDIA Parakeet TDT 0.6b v3, 25 languages) — no audio leaves the browser and nothing is required from the other participants, on either transport.```remote stream → Silero VAD (per participant) → utterance PCM → Worker (Parakeet, WebGPU) → caption overlay```-Each remote mic stream gets a [Silero VAD](https://github.com/ricky0123/vad) finding utterance boundaries; while a speaker keeps talking, the utterance-so-far is re-transcribed every ~1.5s as a provisional line that updates in place, and the speech-end pass finalizes it. Utterances are transcribed by one shared model in a Web Worker (fp32 encoder on WebGPU, int8 decoder on single-threaded WASM — fp16 encoders silently produce empty transcriptions on some GPU stacks, and multithreaded WASM would require COOP/COEP headers). Requires WebGPU; gated by the `voice_subtitles_enabled` site setting.+Each remote mic stream gets a [Silero VAD](https://github.com/ricky0123/vad) finding utterance boundaries; while a speaker keeps talking, the utterance-so-far is re-transcribed every ~1.5s as a provisional line that updates in place, and the speech-end pass finalizes it. Utterances are transcribed by one shared model in a Web Worker (quantized encoder on WebGPU, int8 decoder on single-threaded WASM — multithreaded WASM would require COOP/COEP headers). Requires WebGPU; gated by the `voice_subtitles_enabled` site setting.-The runtime bundles (worker, VAD, onnxruntime, ~41 MB) are pinned and shipped in the discourse_voice_assets gem (served here from `public/javascripts/<gem version>/stt/`, built by the gem's `scripts/build-stt-assets.sh`). The ~2.5 GB model weights are **not** committed: they download on first use from Discourse's HuggingFace repository (kept in a durable Cache API store), or from a self-hosted mirror configured via `voice_stt_model_base_url` — see [docs/subtitles-model-mirror.md](docs/subtitles-model-mirror.md) for what to mirror and how.+Each user picks the model in their voice settings (stored per browser, like the noise-suppression mode; registry in `lib/voice/stt-models.js`): **Best quality** (Parakeet Ultra, 4-bit encoder, ~410 MB, the default) or **Smaller** (Parakeet Redux, ternary encoder stored as 2-bit, ~210 MB, less accurate in noise and in several languages). Switching reloads the worker on the new model, and the worker keeps only the current model in storage.-An end-to-end smoke check (`scripts/smoke-stt-worker.mjs`, real model on WebGPU in headless Chromium) lives in the gem alongside the build script.+The runtime bundles (worker, VAD, onnxruntime, ~30 MB) are pinned and shipped in the discourse_voice_assets gem (served here from `public/javascripts/<gem version>/stt/`, built by the gem's `scripts/build-stt-assets.sh`). The model weights are **not** committed: they download on first use from Discourse's HuggingFace repository ([Discourse/Discourse-STT](https://huggingface.co/Discourse/Discourse-STT), pinned to a commit; kept in a durable Cache API store), or from a self-hosted mirror configured via `voice_stt_model_base_url` — see [docs/subtitles-model-mirror.md](docs/subtitles-model-mirror.md) for what to mirror and how.++An end-to-end smoke check (`scripts/smoke-stt-worker.mjs`, real model on WebGPU in Chromium) lives in the gem alongside the build script.## Developmentdiff --git a/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.js b/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.jsindex 04644e49634..ad4d413b560 100644--- a/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.js+++ b/plugins/voice/assets/javascripts/discourse/app/services/voice-webrtc.js@@ -521,6 +521,10 @@ export default class VoiceWebrtcService extends Service {return this.#transcription.progress;}+ get sttModel() {+ return this.#transcription.sttModel;+ }+get captions() {return this.#transcription.captions;}@@ -633,6 +637,11 @@ export default class VoiceWebrtcService extends Service {this.#transcription.toggle();}+ @action+ setSttModel(model) {+ this.#transcription.setSttModel(model);+ }+isTranscribingRoom(roomId) {return this.#transcription.isTranscribingRoom(roomId);}diff --git a/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjs b/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjsindex 0e7f37c9e86..7b6179ab627 100644--- a/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjs+++ b/plugins/voice/assets/javascripts/discourse/components/modal/voice-voice-settings.gjs@@ -24,6 +24,7 @@ import {engineForMode,noiseSuppressionModeLabel,} from "../../lib/voice/ns-engines";+import { STT_MODELS, sttModelLabel } from "../../lib/voice/stt-models";const METER_INTERVAL_MS = 50;@@ -92,6 +93,13 @@ export default class VoiceVoiceSettingsModal extends Component {}));}+ get sttModelOptions() {+ return Object.keys(STT_MODELS).map((model) => ({+ id: model,+ name: sttModelLabel(model),+ }));+ }+get qualityOptions() {return this.voiceWebrtc.allowedVoiceQualityTiers().map((tier) => ({id: tier,@@ -456,6 +464,22 @@ export default class VoiceVoiceSettingsModal extends Component {{{i18n "voice.voice_settings.subtitles_hint"}}</p></div>++ <div class="voice-voice-settings__field">+ <label class="voice-voice-settings__label">+ {{i18n "voice.voice_settings.subtitles_model"}}+ </label>+ <ComboBox+ class="voice-voice-settings__subtitles-model-select"+ @content={{this.sttModelOptions}}+ @onChange={{this.voiceWebrtc.setSttModel}}+ @options={{hash none=false}}+ @value={{this.voiceWebrtc.sttModel}}+ />+ <p class="voice-voice-settings__hint">+ {{i18n "voice.voice_settings.subtitles_model_hint"}}+ </p>+ </div>{{/if}}{{#if this.showQuality}}diff --git a/plugins/voice/assets/javascripts/discourse/lib/voice/stt-models.js b/plugins/voice/assets/javascripts/discourse/lib/voice/stt-models.jsnew file mode 100644index 00000000000..127fe5b4636--- /dev/null+++ b/plugins/voice/assets/javascripts/discourse/lib/voice/stt-models.js@@ -0,0 +1,57 @@+import { i18n } from "discourse-i18n";++// The selectable speech-to-text models. Each is a folder holding the+// worker's fixed file layout (encoder-model.onnx,+// decoder_joint-model.int8.onnx, vocab.txt) under one model root: a site's+// mirror (voice_stt_model_base_url) or Discourse's HuggingFace repository.+// The default root is pinned to a commit because the worker caches by URL:+// updated weights must arrive under a new URL to ever be downloaded.+//+// Order here is the order shown in the selector.+export const STT_MODELS = {+ ultra: { path: "ultra-q4" },+ redux: { path: "redux-w2a8" },+};++export const DEFAULT_STT_MODEL = "ultra";++const DEFAULT_MODEL_ROOT =+ "https://huggingface.co/Discourse/Discourse-STT/resolve/0862913c8a5c08254d1082747c6443a06e062988";++const STORAGE_KEY = "voice:stt-model";++export function isSttModel(model) {+ return Object.hasOwn(STT_MODELS, model);+}++export function sttModelUrl(model, modelRoot) {+ const root = (modelRoot || DEFAULT_MODEL_ROOT).replace(/\/$/, "");+ return `${root}/${STT_MODELS[model].path}`;+}++export function preferredSttModel() {+ try {+ const model = localStorage.getItem(STORAGE_KEY);+ if (isSttModel(model)) {+ return model;+ }+ } catch {+ // fall through to the default+ }+ return DEFAULT_STT_MODEL;+}++export function setPreferredSttModel(model) {+ if (!isSttModel(model)) {+ return;+ }+ try {+ localStorage.setItem(STORAGE_KEY, model);+ } catch {+ // ignore storage errors+ }+}++export function sttModelLabel(model) {+ return i18n(`voice.voice_settings.subtitles_models.${model}`);+}diff --git a/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.js b/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.jsindex 3dabc99006f..6640de98a31 100644--- a/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.js+++ b/plugins/voice/assets/javascripts/discourse/lib/voice/subtitles.js@@ -4,8 +4,8 @@ import { voiceAssetAppUrl, voiceAssetUrl } from "./voice-assets";// Stable filenames in the discourse_voice_assets gem's stt/ directory.const STT_WORKER_FILE = "subtitles-worker.js";const VAD_BUNDLE_FILE = "vad.js";-const ORT_WASM_JS_FILE = "ort/ort-wasm-simd-threaded.jsep.js";-const ORT_WASM_BINARY_FILE = "ort/ort-wasm-simd-threaded.jsep.wasm";+const ORT_WASM_JS_FILE = "ort/ort-wasm-simd-threaded.asyncify.js";+const ORT_WASM_BINARY_FILE = "ort/ort-wasm-simd-threaded.asyncify.wasm";const VAD_ASSET_DIR = "vad/";// Live subtitles for room participants.@@ -18,9 +18,10 @@ const VAD_ASSET_DIR = "vad/";// Web Worker (WebGPU encoder), which serializes jobs across speakers.//// The runtime bundles are served from the plugin's public dir and only-// fetched the first time subtitles are enabled; the ~2.5GB model weights-// come from the `voice_stt_model_base_url` source (Discourse's-// HuggingFace model repository by default), kept in a durable Cache API store.+// fetched the first time subtitles are enabled; the model weights+// (~200-400MB, see stt-models.js) come from the `voice_stt_model_base_url`+// source (Discourse's HuggingFace model repository by default), kept in a+// durable Cache API store.const PREFERENCE_KEY = "voice:subtitles";@@ -136,7 +137,19 @@ export default class SubtitlesManager {}setEnabled(enabled, { modelBaseUrl = null } = {}) {+ const modelChanged = modelBaseUrl !== this.#modelBaseUrl;this.#modelBaseUrl = modelBaseUrl;+ if (modelChanged && this.#worker) {+ // The worker holds one loaded model, so a different model needs a+ // fresh one. Existing taps keep running and feed it once it's ready.+ this.#terminateWorker();+ if (enabled && this.#taps.size) {+ this.#ensureWorker(modelBaseUrl);+ } else {+ this.#onLoadingChange();+ }+ }+if (this.#enabled === enabled) {return;}@@ -306,7 +319,7 @@ export default class SubtitlesManager {}// Best effort, window-only API: asks the browser to exempt the origin's- // storage (the multi-GB cached model) from eviction. Chromium decides+ // storage (the cached model) from eviction. Chromium decides// silently, but Firefox shows a permission prompt — so this must only// run here, when transcription is actually starting in a room, never on// plain page load.@@ -340,7 +353,7 @@ export default class SubtitlesManager {worker.postMessage({type: "init",config: {- modelBaseUrl: modelBaseUrl || null,+ modelBaseUrl,ortWasmJsUrl: sttUrl(ORT_WASM_JS_FILE),ortWasmBinaryUrl: sttUrl(ORT_WASM_BINARY_FILE),},diff --git a/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.js b/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.jsindex 8ab259e4614..d97cac9e152 100644--- a/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.js+++ b/plugins/voice/assets/javascripts/discourse/lib/voice/transcription-coordinator.js@@ -5,6 +5,12 @@ import Composer from "discourse/models/composer";import Draft from "discourse/models/draft";import { i18n } from "discourse-i18n";import voiceLog from "discourse/plugins/voice/discourse/lib/voice/logger";+import {+ isSttModel,+ preferredSttModel,+ setPreferredSttModel,+ sttModelUrl,+} from "./stt-models";import SubtitlesManager from "./subtitles";import TranscriptDraftSync from "./transcript-draft-sync";import { transcriptToMarkdown } from "./transcript-markdown";@@ -20,6 +26,7 @@ export default class TranscriptionCoordinator {@tracked progress = null;@tracked captions = [];@tracked revision = 0;+ @tracked sttModel = preferredSttModel();#subtitles;#transcript;@@ -119,7 +126,10 @@ export default class TranscriptionCoordinator {}get #modelBaseUrl() {- return this.#siteSettings.voice_stt_model_base_url || null;+ return sttModelUrl(+ this.sttModel,+ this.#siteSettings.voice_stt_model_base_url+ );}get recording() {@@ -158,6 +168,15 @@ export default class TranscriptionCoordinator {this.#syncSttEngine();}+ setSttModel(model) {+ if (!isSttModel(model) || model === this.sttModel) {+ return;+ }+ this.sttModel = model;+ setPreferredSttModel(model);+ this.#syncSttEngine();+ }+isTranscribingRoom(roomId) {return this.recording && Number(this.roomId) === Number(roomId);}diff --git a/plugins/voice/config/locales/client.en.yml b/plugins/voice/config/locales/client.en.ymlindex 898f0511273..50213e36725 100644--- a/plugins/voice/config/locales/client.en.yml+++ b/plugins/voice/config/locales/client.en.yml@@ -172,7 +172,12 @@ en:auto_gain_control: "Automatic gain control"auto_gain_control_hint: "Keeps your volume steady by adjusting the microphone level automatically."subtitles: "Show live subtitles"- subtitles_hint: "Generates captions for what others say, on your device — nothing is sent anywhere. Requires a GPU-capable browser and downloads a ~2.5 GB model once."+ subtitles_hint: "Generates captions for what others say, on your device — nothing is sent anywhere. Requires a GPU-capable browser and downloads the subtitles model once."+ subtitles_model: "Subtitles model"+ subtitles_model_hint: "Best quality is the most accurate. Smaller downloads about half as much, but makes more mistakes in noisy rooms and in some languages."+ subtitles_models:+ ultra: "Best quality (~410 MB)"+ redux: "Smaller (~210 MB)"subtitles_failed: "Subtitles couldn't be started, so they were turned off."test_output: "Test"choose_output: "Choose output device"diff --git a/plugins/voice/config/locales/server.en.yml b/plugins/voice/config/locales/server.en.ymlindex d3df172f6f5..ced29985c6a 100644--- a/plugins/voice/config/locales/server.en.yml+++ b/plugins/voice/config/locales/server.en.yml@@ -22,8 +22,8 @@ en:voice_screen_share_allowed_groups: "Groups whose members can share their screen in voice rooms. Leave empty to turn screen sharing off site-wide. Rooms can individually opt out of all media via their room settings, and stage room listeners can never publish."voice_video_max_publishers: "Maximum number of participants who can share video or their screen at the same time in a room. On peer-to-peer rooms, higher values increase CPU and bandwidth load for everyone watching; on rooms using a LiveKit media server, they mainly increase load on that server."voice_video_background_blur_enabled: "Allow participants to blur their camera background. Segmentation runs on the publisher's device via MediaPipe, which increases CPU/GPU load while their camera is on."- voice_subtitles_enabled: "Allow participants to turn on live subtitles for other people's speech. Transcription runs entirely on the viewer's device via an NVIDIA Parakeet speech-to-text model (~2.5 GB, downloaded on first use, WebGPU required); no audio leaves the browser."- voice_stt_model_base_url: "Optional base URL for a self-hosted mirror of the subtitles model. When blank, the model files are downloaded on first use from Discourse's HuggingFace repository (Discourse/parakeet-tdt-0.6b-v3-onnx). A mirror must serve the same four files under their original names, reachable by browsers with CORS allowed — see docs/subtitles-model-mirror.md in the plugin."+ voice_subtitles_enabled: "Allow participants to turn on live subtitles for other people's speech. Transcription runs entirely on the viewer's device via an NVIDIA Parakeet speech-to-text model (less than 0.5 GB, downloaded on first use, WebGPU required); no audio leaves the browser."+ voice_stt_model_base_url: "Optional base URL for a self-hosted mirror of the subtitles models. When blank, the model files are downloaded on first use from Discourse's HuggingFace repository (Discourse/Discourse-STT). A mirror must keep that repository's folder layout and file names, reachable by browsers with CORS allowed — see docs/subtitles-model-mirror.md in the plugin."voice_max_voice_quality: "Highest microphone audio quality participants may choose in their voice settings. Standard is the browser's speech default (~32 kbps); high (~64 kbps) and maximum (~128 kbps) improve fidelity for good microphones at the cost of bandwidth."voice_max_camera_quality: "Highest camera quality participants may choose in their video settings. Standard captures at 720p; high captures at 1080p; maximum additionally stops automatic quality reduction as viewer count grows on peer-to-peer rooms, which multiplies the sender's upload bandwidth and CPU use."voice_max_screen_share_quality: "Highest screen share quality participants may choose in their video settings. Standard targets 15 fps at a modest bitrate; high allows 30 fps at a higher bitrate; maximum allows 60 fps at a very high bitrate ceiling, letting fast connections share high-resolution screens at full quality."diff --git a/plugins/voice/docs/subtitles-fp16-encoder.md b/plugins/voice/docs/subtitles-fp16-encoder.mddeleted file mode 100644index 2c7dd45c463..00000000000--- a/plugins/voice/docs/subtitles-fp16-encoder.md+++ /dev/null@@ -1,166 +0,0 @@-# Subtitles: fp16 encoder--Status (2026-08-26): **auto-selected and validated.** The worker picks fp16-when the adapter exposes `shader-f16` and fp32 otherwise (Chromium on-Linux). Root cause of the historical empty-string failure is diagnosed-below. The fp16 path is validated end-to-end on real f16 hardware via-Firefox on Linux (`STT_BROWSER=firefox` in the smoke harness), which —-unlike Chromium — exposes `shader-f16` there.--## Why we want it--The Parakeet encoder runs on WebGPU in fp32 today:--- Weights: `encoder-model.onnx.data` is ~2.4 GB in fp32; an fp16 encoder is- ~1.2 GB — half the download, disk cache, and resident VRAM.-- Activations: onnxruntime-web's WebGPU execution provider keeps every freed- GPU buffer in a size-keyed free list for the lifetime of the session (no- eviction, no return to the driver). Since utterances arrive in many- distinct lengths, the pool converges to the union of all activation-buffer- sizes ever seen — observed growing from ~3.2 GB to ~5.5 GB over an hour of- live captioning. fp16 would halve every one of those buffers, on top of- halving the weights.--On low-VRAM devices (e.g. 8 GB shared with the compositor and video decode)-the fp32 footprint risks driver paging (massive slowdown) and eventually-WebGPU device loss.--## Symptom--With an fp16 encoder, transcription **silently returns empty strings** on-some machines — no error, no rejected promise, just blank output. The-upstream demo (<https://ysdede.github.io/keet/>) shows the same failure-here. This is noted in `src/stt-worker/worker.js` next to the-`encoderQuant` default.--## Root cause (diagnosed 2026-08-25)--Three findings, established empirically on a Linux machine that reproduces-the failure (RTX 5090, NVIDIA 610.57.04, Chromium 151):--1. **Chromium on Linux does not expose the WebGPU `shader-f16` feature at- all** — on any GPU. The Vulkan driver advertises `shaderFloat16` and- 16-bit storage, but the adapter's feature set lacks `shader-f16`, and no- flag (`--enable-dawn-features`, `--enable-blink-features`,- `--enable-webgpu-developer-features`) turns it on. Per the Chromium- intent-to-ship for WebGPU f16, the feature shipped on macOS/ChromeOS- first and on Windows behind the DXC migration; Linux never got it.- "Works on some machines, fails on others" is **feature availability by- platform**, not driver numerics. (Firefox's WebGPU on the same- machine/driver _does_ expose `shader-f16` — the gap is Chromium's, not- Vulkan's.)-2. **onnxruntime-web runs fp16 models anyway when the feature is missing.**- The JSEP WebGPU EP requests `shader-f16` on the device and prepends- `enable f16;` to WGSL _only if the adapter has it_, but never refuses an- fp16 model when it doesn't — execution proceeds and produces garbage- encodings, the decoder emits nothing, and no error surfaces. (Upstream- bug worth filing: this should be a hard error or a fallback.)-3. **The fp16 exports themselves are numerically fine.** On the CPU EP,- both `grikdotnet/parakeet-tdt-0.6b-fp16` (full fp16, fp32 IO) and our- own mixed-precision exports match the fp32 encoder to cosine similarity- ≥ 0.999998 with zero NaN/Inf.--The earlier overflow hypothesis is **disproved** for this model: scanning-all float intermediates of the fp32 encoder over speech fixtures-(`scripts/scan_overflow.py`) shows a global max |activation| of exactly-10000.0 (the attention-mask fill constant) — far inside fp16's 65504 range.-The transformers.js Whisper issue we cited-(<https://github.com/huggingface/transformers.js/issues/1590>) is a-different failure class.--Community fp16 exports, for the record: `grikdotnet/parakeet-tdt-0.6b-fp16`-(numerically clean, CUDA-validated only) and-`Olicorne/parakeet-tdt-0.6b-v3-optimized-onnx` (fp16 files were a naive-cast, withdrawn by the author 2026-08-23; repo remains interesting for its-sharded fp32 encoder). Neither fails because of how it was exported — any-fp16 export fails wherever `shader-f16` is unavailable.--## Current behavior--The worker (`src/stt-worker/worker.js`) resolves the encoder precision at-init: an explicit `encoderQuant` of `"fp16"`/`"fp32"` is honored (the smoke-harness forces it via `STT_ENCODER_QUANT`), anything else auto-selects via-`adapter.features.has("shader-f16")`. fp16 loads the self-contained-`encoder-model.fp16.onnx` from our model repo (fp32 IO, so parakeet.js's-float32 feeds work unchanged); fp32 keeps the sidecar pair. Today that-means fp16 for macOS/ChromeOS/newer-Windows Chrome, fp32 for all of Linux.--Verified both ways on the same machine (RTX 5090):--- Chromium (no `shader-f16`): auto picks fp32 and transcribes; forced fp16- reproduces the empty-string failure, as expected.-- Firefox (`shader-f16` exposed): auto picks fp16 and the full smoke- harness passes — correct transcription and VAD utterances with the- shipped `encoder-model.fp16.onnx`.--## Still open--1. Consider an init warm-up canary (a fixture must transcribe to non-empty- text, else re-init as fp32) as a belt-and-braces fallback for genuinely- broken f16 drivers.-2. Watch for Chromium enabling `shader-f16` on Linux, and consider filing- the ORT-web silent-execution bug upstream.-3. The f16 kernels are only exercised on Firefox/Vulkan so far; Chromium's- Metal/D3D backends (where it will actually run fp16 in production) run- the same ORT WGSL but haven't been checked by us directly.--Note: int8/fp8 are not alternatives for the encoder — WGSL has no 8-bit-types (only packed DP4a-style dot products), ORT-web falls back off the-WebGPU EP for quantized encoder ops (parakeet.js upgrades int8 encoder-requests to fp32 on WebGPU backends), and fp8 does not exist in WebGPU at-all. The int8 decoder already runs on WASM, unaffected.--## Reproducing the diagnosis--Checking any adapter takes one line in a devtools console:--```js-(await navigator.gpu.requestAdapter()).features.has("shader-f16");-```--The numeric findings came from running candidate encoders on-onnxruntime's CPU EP against the fp32 reference (same mel input, compare-outputs for NaN/Inf and cosine similarity), and from running the fp32-encoder with all intermediates exposed as graph outputs to scan activation-ranges. Mixed-precision exports were produced with onnxconverter-common's-`convert_float_to_float16` (`keep_io_types=True`); none of that tooling is-needed going forward, since the shipped fp16 file already exists in the-model repo.--The smoke harness (`scripts/smoke-stt-worker.mjs` in the-discourse_voice_assets gem repository) now logs the adapter and-launches Chromium with `Vulkan,VulkanFromANGLE,DefaultANGLEVulkan` —-without the FromANGLE pair, headless Chromium silently falls back to-SwiftShader (CPU rasterizer, no `shader-f16`, ~1000× slower encode), which-is both a useless benchmark and an accidental repro of the no-f16 failure.--## How to test--The smoke harness exercises the real worker in headless Chromium; point it-at a directory containing candidate files (the worker fetches-`encoder-model.onnx`, `encoder-model.onnx.data` — a zero-byte file works-for self-contained encoders — `decoder_joint-model.int8.onnx`, `vocab.txt`):--```bash-flite -t "the quick brown fox jumps over the lazy dog" /tmp/fix.wav-STT_MODEL_DIR=/path/to/model-dir node scripts/smoke-stt-worker.mjs /tmp/fix.wav-```--Check the `adapter:` line in the output: on an adapter without-`shader-f16`, an fp16-encoder FAIL only reproduces the availability bug and-says nothing about the export's numerics. Use `STT_SMOKE_PORT` to give each-model dir its own origin, since downloads are cached per-origin in the-persistent profile.--To exercise the fp16 path on Linux, run the harness in Firefox — the only-browser exposing `shader-f16` here (WebGPU is enabled via prefs in a-separate persistent profile):--```bash-STT_BROWSER=firefox STT_MODEL_DIR=/path/to/model-dir \- node scripts/smoke-stt-worker.mjs /tmp/fix.wav-```--The model dir needs `encoder-model.fp16.onnx` (self-contained; a zero-byte-`encoder-model.onnx.data` still satisfies the fp32 branch if you force it)-— auto-select picks fp16 there and the whole chain should PASS.diff --git a/plugins/voice/docs/subtitles-model-mirror.md b/plugins/voice/docs/subtitles-model-mirror.mdindex c26ab6fa477..341078cd0cf 100644--- a/plugins/voice/docs/subtitles-model-mirror.md+++ b/plugins/voice/docs/subtitles-model-mirror.md@@ -1,12 +1,13 @@-# Self-hosting the subtitles model (mirror)--By default, live subtitles download the speech-to-text model on first use-from Discourse's HuggingFace repository-([Discourse/parakeet-tdt-0.6b-v3-onnx](https://huggingface.co/Discourse/parakeet-tdt-0.6b-v3-onnx),-a clone of the upstream `ysdede/parakeet-tdt-0.6b-v3-onnx` export). Sites-that can't (or don't want to) let browsers reach `huggingface.co` —-airgapped deployments, enterprise networks, or anyone who prefers to-control the bytes — can serve the model themselves and point the+# Self-hosting the subtitles models (mirror)++By default, live subtitles download their speech-to-text model on first use+from Discourse's HuggingFace repository,+[Discourse/Discourse-STT](https://huggingface.co/Discourse/Discourse-STT).+The plugin pins that repository to a specific commit.++Some sites can't let browsers reach `huggingface.co`, or prefer not to:+airgapped deployments, enterprise networks, or anyone who wants to control+the bytes. They can serve the models themselves and point the`voice_stt_model_base_url` site setting at their own mirror.Only the **model weights** are affected. The runtime bundles (worker, VAD,@@ -15,79 +16,102 @@ need no extra hosting.## What to mirror-Four files from the-[ysdede/parakeet-tdt-0.6b-v3-onnx](https://huggingface.co/ysdede/parakeet-tdt-0.6b-v3-onnx)-repository (models are CC-BY-4.0; keep the repo's attribution requirements in-mind), **keeping their exact filenames**, in a single flat directory:+The repository has one folder per model. Each user picks a model in their+voice settings, so mirror every folder. **Keep the folder names and file+names exactly as they are.**-| File | Size | Purpose |+| Folder | Model | Size || --- | --- | --- |-| `encoder-model.onnx` | ~42 MB | fp32 encoder graph (runs on WebGPU) |-| `encoder-model.onnx.data` | ~2.4 GB | encoder weights (external data referenced by the graph — the filename must not change) |-| `decoder_joint-model.int8.onnx` | ~18 MB | int8 decoder/joint (runs on WASM) |-| `vocab.txt` | ~100 KB | tokenizer vocabulary |+| `ultra-q4/` | Parakeet Ultra, 4-bit encoder (default, "Best quality") | ~410 MB |+| `redux-w2a8/` | Parakeet Redux, 2-bit encoder ("Smaller") | ~210 MB |++Every folder holds the same three files:++| File | Purpose |+| --- | --- |+| `encoder-model.onnx` | quantized encoder, a single file (runs on WebGPU) |+| `decoder_joint-model.int8.onnx` | int8 decoder/joint (runs on WASM) |+| `vocab.txt` | tokenizer vocabulary |++To download them, using the commit the plugin pins (see `DEFAULT_MODEL_ROOT`+in `assets/javascripts/discourse/lib/voice/stt-models.js`):```bash-mkdir parakeet-tdt-0.6b-v3-cd parakeet-tdt-0.6b-v3-for f in encoder-model.onnx encoder-model.onnx.data \- decoder_joint-model.int8.onnx vocab.txt; do- curl -LO "https://huggingface.co/ysdede/parakeet-tdt-0.6b-v3-onnx/resolve/main/$f"+REV=0862913c8a5c08254d1082747c6443a06e062988+for model in ultra-q4 redux-w2a8; do+ mkdir -p "discourse-stt/$model"+ for f in encoder-model.onnx decoder_joint-model.int8.onnx vocab.txt; do+ curl -L -o "discourse-stt/$model/$f" \+ "https://huggingface.co/Discourse/Discourse-STT/resolve/$REV/$model/$f"+ donedone```-No subdirectory structure is required or supported: the plugin requests-`<base_url>/<filename>` for each of the four files, nothing else.+The models are CC-BY-4.0. The repository's README lists the attribution and+the source of every file.++The plugin requests `<base_url>/<folder>/<file>` and nothing else.## Hosting requirements-The files are fetched by the **browser** (from a Web Worker on the forum's-origin), not by the Discourse server, so the mirror must be reachable by your-users and must speak CORS if it lives on a different origin:+The files are fetched by the **browser**, from a Web Worker on the forum's+origin, not by the Discourse server. So the mirror must be reachable by your+users, and must speak CORS if it lives on a different origin:-- `Access-Control-Allow-Origin: <forum origin>` (or `*`) on all four files.+- `Access-Control-Allow-Origin: <forum origin>` (or `*`) on every file.- HTTPS, if the forum is served over HTTPS (mixed content is blocked).-- Correct `Content-Length` (any static file server does this) — it drives- the download progress shown in the caption overlay, and the ~2.4 GB file- arrives as one plain GET; resumable/range support is not needed.-- Long-lived cache headers are still recommended (the files are immutable),- though the browser keeps a durable Cache API copy after the first- download, so mirrors are only re-hit when that storage is evicted.+- A correct `Content-Length`, which any static file server sends. It drives+ the download progress shown in the caption overlay. Range requests are+ not needed.+- Long-lived cache headers are recommended, because the files never change.Example nginx location:```nginx-location /models/parakeet-tdt-0.6b-v3/ {+location /models/discourse-stt/ {root /var/www;add_header Access-Control-Allow-Origin "https://forum.example.com";add_header Cache-Control "public, max-age=31536000, immutable";}```-Then set the site setting:+Then set the site setting (the trailing slash is optional):```-voice_stt_model_base_url = https://cdn.example.com/models/parakeet-tdt-0.6b-v3+voice_stt_model_base_url = https://cdn.example.com/models/discourse-stt```-(Trailing slash optional.)+## Updating a mirror++Browsers keep a durable copy of the model they use and look it up by URL.+If you replace a file at the same URL, users who already have the old copy+keep it.-A custom mirror behaves exactly like the default source: streamed download-progress in the caption overlay and a durable Cache API copy on the user's-device.+When the plugin moves to a new model revision, publish the new files under a+new directory and point the setting there. For example, put the revision in+the path: `.../discourse-stt/<rev>/`.++Each browser keeps only one model. Switching models, or changing the mirror+URL, frees the previous copy.## Verifying a mirror-The end-to-end smoke harness (in the discourse_voice_assets gem repository,-alongside the shipped worker bundle) can run against a local copy of the-mirror directory:+The end-to-end smoke harness lives in the discourse_voice_assets gem+repository, alongside the shipped worker bundle. It can run against a local+copy of any one model folder:```bashflite -t "the quick brown fox jumps over the lazy dog" /tmp/fix.wav-STT_MODEL_DIR=/path/to/parakeet-tdt-0.6b-v3 \+STT_MODEL_DIR=/path/to/discourse-stt/ultra-q4 \+ node scripts/smoke-stt-worker.mjs /tmp/fix.wav+```++Or point it at the mirror itself:++```bash+STT_MODEL_BASE_URL=https://cdn.example.com/models/discourse-stt/ultra-q4 \node scripts/smoke-stt-worker.mjs /tmp/fix.wav```-It loads the shipped worker in headless Chromium (WebGPU required), serves-the model files from the given directory exactly as a mirror would, and-fails unless the fixture transcribes.+The harness loads the shipped worker in Chromium (WebGPU required) and fails+unless the fixture transcribes.diff --git a/plugins/voice/spec/integrity/voice_assets_spec.rb b/plugins/voice/spec/integrity/voice_assets_spec.rbindex 6f91d897851..af9735143bf 100644--- a/plugins/voice/spec/integrity/voice_assets_spec.rb+++ b/plugins/voice/spec/integrity/voice_assets_spec.rb@@ -19,8 +19,8 @@ RSpec.describe "discourse_voice_assets integrity" dornnoise/rnnoise.wasmstt/subtitles-worker.jsstt/vad.js- stt/ort/ort-wasm-simd-threaded.jsep.js- stt/ort/ort-wasm-simd-threaded.jsep.wasm+ stt/ort/ort-wasm-simd-threaded.asyncify.js+ stt/ort/ort-wasm-simd-threaded.asyncify.wasmstt/vad/livekit/livekit-client.jsmediapipe/vision_bundle.jsdiff --git a/plugins/voice/spec/system/voice_subtitles_spec.rb b/plugins/voice/spec/system/voice_subtitles_spec.rbindex 6feb048f6b4..976eeae72e2 100644--- a/plugins/voice/spec/system/voice_subtitles_spec.rb+++ b/plugins/voice/spec/system/voice_subtitles_spec.rb@@ -69,4 +69,20 @@ describe "Voice subtitles" dofind(".modal-close").clickexpect(page).to have_css(".voice-captions", visible: :all)end++ it "lets the user pick the smaller subtitles model" do+ SiteSetting.voice_subtitles_enabled = true+ stub_webgpu+ join_room+ open_voice_settings++ select = PageObjects::Components::SelectKit.new(".voice-voice-settings__subtitles-model-select")+ expect(select).to have_selected_name(I18n.t("js.voice.voice_settings.subtitles_models.ultra"))++ select.expand+ select.select_row_by_value("redux")++ expect(select).to have_selected_name(I18n.t("js.voice.voice_settings.subtitles_models.redux"))+ expect(page.evaluate_script("localStorage.getItem('voice:stt-model')")).to eq("redux")+ endenddiff --git a/plugins/voice/test/javascripts/unit/lib/stt-models-test.js b/plugins/voice/test/javascripts/unit/lib/stt-models-test.jsnew file mode 100644index 00000000000..62b36747c0b--- /dev/null+++ b/plugins/voice/test/javascripts/unit/lib/stt-models-test.js@@ -0,0 +1,59 @@+import { setupTest } from "ember-qunit";+import { module, test } from "qunit";+import {+ DEFAULT_STT_MODEL,+ preferredSttModel,+ setPreferredSttModel,+ STT_MODELS,+ sttModelUrl,+} from "discourse/plugins/voice/discourse/lib/voice/stt-models";++module("Voice | Unit | Lib | stt-models", function (hooks) {+ setupTest(hooks);++ hooks.beforeEach(function () {+ localStorage.removeItem("voice:stt-model");+ });++ hooks.afterEach(function () {+ localStorage.removeItem("voice:stt-model");+ });++ test("defaults to the best-quality model", function (assert) {+ assert.strictEqual(DEFAULT_STT_MODEL, "ultra");+ assert.strictEqual(preferredSttModel(), "ultra");+ });++ test("remembers a chosen model and ignores unknown ones", function (assert) {+ setPreferredSttModel("redux");+ assert.strictEqual(preferredSttModel(), "redux");++ setPreferredSttModel("v3");+ assert.strictEqual(+ preferredSttModel(),+ "redux",+ "unknown ids aren't stored"+ );++ localStorage.setItem("voice:stt-model", "v3");+ assert.strictEqual(+ preferredSttModel(),+ "ultra",+ "a stale stored id falls back to the default"+ );+ });++ test("model URLs are folders under the pinned repository or a mirror", function (assert) {+ assert.true(+ /^https:\/\/huggingface\.co\/Discourse\/Discourse-STT\/resolve\/[0-9a-f]{40}\/ultra-q4$/.test(+ sttModelUrl("ultra")+ ),+ "the default root is pinned to a commit"+ );+ assert.strictEqual(+ sttModelUrl("redux", "https://cdn.example.com/stt/"),+ "https://cdn.example.com/stt/redux-w2a8"+ );+ assert.deepEqual(Object.keys(STT_MODELS), ["ultra", "redux"]);+ });+});diff --git a/plugins/voice/test/javascripts/unit/lib/subtitles-test.js b/plugins/voice/test/javascripts/unit/lib/subtitles-test.jsindex 26bec43a9b4..5783d7fad21 100644--- a/plugins/voice/test/javascripts/unit/lib/subtitles-test.js+++ b/plugins/voice/test/javascripts/unit/lib/subtitles-test.js@@ -351,4 +351,54 @@ module("Voice | Unit | Lib | subtitles", function (hooks) {assert.strictEqual(job.userId, 9);assert.strictEqual(job.pcm.byteLength, 8000 * 4);});++ test("the worker is initialized with the chosen model's URL", async function (assert) {+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/ultra" });+ await this.manager.attach(1, 42, createFakeStream("s1"));++ const init = FakeWorker.instances[0].messages.find(+ (message) => message.type === "init"+ );+ assert.strictEqual(init.config.modelBaseUrl, "https://models/ultra");+ assert.true(+ init.config.ortWasmBinaryUrl.endsWith(+ "stt/ort/ort-wasm-simd-threaded.asyncify.wasm"+ )+ );+ });++ test("switching models replaces the worker and keeps the taps", async function (assert) {+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/ultra" });+ await this.manager.attach(1, 42, createFakeStream("s1"));+ await new Promise((resolve) => setTimeout(resolve, 0));++ this.manager.setEnabled(true, { modelBaseUrl: "https://models/redux" });++ assert.true(FakeWorker.instances[0].terminated, "old worker terminated");+ assert.strictEqual(FakeWorker.instances.length, 2, "new worker started");+ assert.strictEqual(+ FakeWorker.instances[1].messages[0].config.modelBaseUrl,+ "https://models/redux"+ );+ assert.false(FakeVad.instances[0].destroyed, "the tap survives");++ await new Promise((resolve) => setTimeout(resolve, 0));+ FakeVad.instances[0].speak();+ await new Promise((resolve) => setTimeout(resolve, 10));++ assert.strictEqual(FakeWorker.instances[1].jobs.length, 1);+ assert.strictEqual(this.captions.length, 1, "captions keep flowing");+ });++ test("switching models without taps defers the new worker", async function (assert) {+ this.manager.setEnabled(true, { modelBaseUrl: "https://models/ultra" });+ await this.manager.attach(1, 42, createFakeStream("s1"));+ this.manager.detachRoom(1);++ this.manager.setEnabled(true, { modelBaseUrl: "https://models/redux" });++ assert.true(FakeWorker.instances[0].terminated);+ assert.strictEqual(FakeWorker.instances.length, 1, "nothing to transcribe");+ assert.false(this.manager.loading);+ });});