Skip to content

Build your personal knowledge base with Trilium Notes

chore(standalone): prototype in-browser transcription with sherpa-onnx WebAssembly

Elian Doran41d264d71d88
diff --git a/apps/standalone/transcription-poc/.gitignore b/apps/standalone/transcription-poc/.gitignore
new file mode 100644
index 0000000000..48b8bf9072
--- /dev/null
+++ b/apps/standalone/transcription-poc/.gitignore
@@ -0,0 +1 @@
+vendor/
diff --git a/apps/standalone/transcription-poc/README.md b/apps/standalone/transcription-poc/README.md
new file mode 100644
index 0000000000..391d29739a
--- /dev/null
+++ b/apps/standalone/transcription-poc/README.md
@@ -0,0 +1,42 @@
+# Transcription in the browser (prototype)
+
+Checks whether meeting transcription can run locally in standalone, with the sherpa-onnx WebAssembly
+build instead of the native `sherpa-onnx-node` addon desktop uses. Not part of the standalone build:
+it sits outside Vite's `src` root and is served by its own server.
+
+```bash
+node apps/standalone/transcription-poc/fetch-assets.mjs # once: the wasm build and Silero VAD into vendor/
+node apps/standalone/transcription-poc/serve.mjs # http://localhost:5555/
+```
+
+Pick a model, *Download to OPFS* (from Hugging Face, pinned revisions), *Load*, then *Transcribe clip* or
+*Listen to the microphone*. Timings and the WebAssembly memory size go to the log.
+
+## How it works
+
+- **The build.** The npm `sherpa-onnx` package is compiled with `NODERAWFS` and throws outside Node, and
+ the GitHub releases carry no browser build of the recognizer. The only one published is the
+ `web-assembly-vad-asr-sherpa-onnx-zh-zipformer-ctc` Hugging Face space. Its binary contains every offline
+ model type (NeMo transducers included); `fetch-assets.mjs` patches its loader so the baked-in 368 MB
+ Chinese model is never fetched.
+- **The models** stream from Hugging Face (which sends CORS headers) into OPFS, in a worker. Loading copies
+ each file into Emscripten's MEMFS with `FS_createDataFile(..., canOwn)`, creates the recognizer, then
+ unlinks the files.
+- **Single-threaded.** Standalone sends `Cross-Origin-Opener-Policy` without
+ `Cross-Origin-Embedder-Policy`, so it is not cross-origin isolated, and this build has no threads anyway.
+- **The memory ceiling** is fixed by the build: 512 MiB initial, 2 GiB maximum.
+
+## Results
+
+Headless Chromium 153, Linux, 24 cores (one used). RTF is decode time over audio length.
+
+| Model | Download | Load from OPFS | WebAssembly memory | Tab RSS | RTF | English | French | German |
+|---|---|---|---|---|---|---|---|---|
+| Parakeet TDT 0.6B v3 int8 | 670 MB | 1.3 s | 1,340 MiB | ~2.4 GiB | 0.22–0.25 | correct | correct | correct |
+| SenseVoice Small int8 | 237 MB | 0.7 s | 609 MiB | ~1.2 GiB | ~0.11 | uppercase, small errors | empty | garbled |
+| Moonshine base int8 | 287 MB | 0.9 s | 512 MiB | ~1.2 GiB | 0.05–0.10 | correct | translated to English | "1." |
+
+Live microphone (Chromium's fake device playing the 17 s English clip, Silero VAD, 8 s maximum segment):
+Parakeet delivers each utterance about 2.2 s after it ends, at RTF 0.22–0.26.
+
+Not measured: Firefox and Safari (Playwright's Firefox does not start on NixOS), phones, slower CPUs.
diff --git a/apps/standalone/transcription-poc/fetch-assets.mjs b/apps/standalone/transcription-poc/fetch-assets.mjs
new file mode 100644
index 0000000000..190bcbdaf6
--- /dev/null
+++ b/apps/standalone/transcription-poc/fetch-assets.mjs
@@ -0,0 +1,39 @@
+// Downloads the prebuilt sherpa-onnx WebAssembly build and the Silero VAD model into `vendor/`.
+//
+// The build comes from k2-fsa's VAD + ASR demo space, the only browser build of the recognizer
+// they publish. It bakes a Chinese model into a preloaded `.data` package; the loader call is
+// patched to an empty package, since the page writes its own models into the virtual FS.
+import { createWriteStream } from "node:fs";
+import { mkdir, readFile, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import { Readable } from "node:stream";
+import { pipeline } from "node:stream/promises";
+
+const SPACE = "https://huggingface.co/spaces/k2-fsa/web-assembly-vad-asr-sherpa-onnx-zh-zipformer-ctc/resolve/9d5fc71d88ab1222ed1af2274fb194592af48455";
+const VAD_URL = "https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx";
+const GLUE = "sherpa-onnx-wasm-main-vad-asr.js";
+const VENDOR_DIR = join(import.meta.dirname, "vendor");
+
+await mkdir(VENDOR_DIR, { recursive: true });
+for (const file of [ GLUE, "sherpa-onnx-wasm-main-vad-asr.wasm", "sherpa-onnx-asr.js", "sherpa-onnx-vad.js" ]) {
+ await download(`${SPACE}/${file}`, join(VENDOR_DIR, file));
+}
+await download(VAD_URL, join(VENDOR_DIR, "silero_vad.onnx"));
+
+const gluePath = join(VENDOR_DIR, GLUE);
+const glue = await readFile(gluePath, "utf8");
+const patched = glue.replace(/loadPackage\(\{"files":\[.*?\],"remote_package_size":\d+\}\)/, `loadPackage({"files":[],"remote_package_size":0})`);
+if (patched === glue) {
+ throw new Error(`The data package loader was not found in ${GLUE}.`);
+}
+await writeFile(gluePath, patched);
+console.log("Done.");
+
+async function download(url, destination) {
+ console.log(`Downloading ${url}`);
+ const response = await fetch(url);
+ if (!response.ok || !response.body) {
+ throw new Error(`${url}: HTTP ${response.status}`);
+ }
+ await pipeline(Readable.fromWeb(response.body), createWriteStream(destination));
+}
diff --git a/apps/standalone/transcription-poc/index.html b/apps/standalone/transcription-poc/index.html
new file mode 100644
index 0000000000..2efd497fce
--- /dev/null
+++ b/apps/standalone/transcription-poc/index.html
@@ -0,0 +1,37 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Transcription prototype</title>
+ <style>
+ body { font-family: system-ui, sans-serif; margin: 16px; max-width: 960px; }
+ fieldset { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; margin-bottom: 12px; }
+ #progress { color: #666; font-variant-numeric: tabular-nums; }
+ #log { white-space: pre-wrap; background: #f4f4f4; padding: 12px; min-height: 200px; font-size: 13px; }
+ </style>
+</head>
+<body>
+ <h1>Transcription prototype (sherpa-onnx WebAssembly)</h1>
+
+ <fieldset>
+ <legend>Model</legend>
+ <select id="model"></select>
+ <button id="download">Download to OPFS</button>
+ <button id="load">Load</button>
+ <button id="delete">Delete from OPFS</button>
+ <span id="progress"></span>
+ </fieldset>
+
+ <fieldset>
+ <legend>Transcribe</legend>
+ <select id="clip"></select>
+ <button id="transcribe">Transcribe clip</button>
+ <button id="listen">Listen to the microphone</button>
+ </fieldset>
+
+ <pre id="log"></pre>
+
+ <script src="main.js"></script>
+</body>
+</html>
diff --git a/apps/standalone/transcription-poc/main.js b/apps/standalone/transcription-poc/main.js
new file mode 100644
index 0000000000..f2a6001d33
--- /dev/null
+++ b/apps/standalone/transcription-poc/main.js
@@ -0,0 +1,190 @@
+// The page side of the prototype: the model catalog, the controls, clip decoding and microphone
+// capture. Everything heavy happens in worker.js.
+const HF = "https://huggingface.co";
+
+/** The candidate models, pinned to a revision. `modelConfig` paths are where worker.js puts the files. */
+const MODELS = [
+ {
+ id: "parakeet-tdt-0.6b-v3-int8",
+ label: "Parakeet TDT 0.6B v3 (int8, 25 European languages)",
+ repo: "csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8",
+ revision: "2bda32ec70b097a55adaa07d9a7173915b43cc78",
+ files: [
+ [ "encoder.int8.onnx", 652184281 ],
+ [ "decoder.int8.onnx", 11845275 ],
+ [ "joiner.int8.onnx", 6355277 ],
+ [ "tokens.txt", 93939 ]
+ ],
+ modelConfig: (dir) => ({
+ transducer: {
+ encoder: `${dir}/encoder.int8.onnx`,
+ decoder: `${dir}/decoder.int8.onnx`,
+ joiner: `${dir}/joiner.int8.onnx`
+ },
+ tokens: `${dir}/tokens.txt`,
+ modelType: "nemo_transducer"
+ })
+ },
+ {
+ id: "sense-voice-int8-2025-09-09",
+ label: "SenseVoice Small (int8, zh/en/ja/ko/yue)",
+ repo: "csukuangfj/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2025-09-09",
+ revision: "355f4d4884d8afd08aef04b9007a8556d7b463b2",
+ files: [
+ [ "model.int8.onnx", 237115547 ],
+ [ "tokens.txt", 315894 ]
+ ],
+ modelConfig: (dir) => ({
+ senseVoice: { model: `${dir}/model.int8.onnx`, language: "auto", useInverseTextNormalization: 1 },
+ tokens: `${dir}/tokens.txt`
+ })
+ },
+ {
+ id: "moonshine-base-en-int8",
+ label: "Moonshine base (int8, English)",
+ repo: "csukuangfj/sherpa-onnx-moonshine-base-en-int8",
+ revision: "052b0798ad1bf046a140fdd4efcd9426530fa3f5",
+ files: [
+ [ "preprocess.onnx", 14077290 ],
+ [ "encode.int8.onnx", 50311494 ],
+ [ "uncached_decode.int8.onnx", 122120451 ],
+ [ "cached_decode.int8.onnx", 99983837 ],
+ [ "tokens.txt", 436688 ]
+ ],
+ modelConfig: (dir) => ({
+ moonshine: {
+ preprocessor: `${dir}/preprocess.onnx`,
+ encoder: `${dir}/encode.int8.onnx`,
+ uncachedDecoder: `${dir}/uncached_decode.int8.onnx`,
+ cachedDecoder: `${dir}/cached_decode.int8.onnx`
+ },
+ tokens: `${dir}/tokens.txt`
+ })
+ }
+];
+
+const CLIPS = [
+ [ "English, 17 s", `${HF}/csukuangfj/sherpa-onnx-moonshine-base-en-int8/resolve/052b0798ad1bf046a140fdd4efcd9426530fa3f5/test_wavs/1.wav` ],
+ [ "French, 7 s", `${HF}/csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3/resolve/1a468a35cbba69418f126de829e75261dea4a4e4/test_wavs/fr.wav` ],
+ [ "German, 4 s", `${HF}/csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3/resolve/1a468a35cbba69418f126de829e75261dea4a4e4/test_wavs/de.wav` ]
+];
+
+const $ = (id) => document.getElementById(id);
+const worker = new Worker("worker.js");
+let microphone = null;
+
+for (const model of MODELS) {
+ $("model").append(new Option(model.label, model.id));
+}
+for (const [ label ] of CLIPS) {
+ $("clip").append(new Option(label, label));
+}
+
+worker.onmessage = ({ data }) => {
+ switch (data.type) {
+ case "ready": return log(`Runtime ready, WebAssembly memory ${data.heap} MiB. ${environment()}`);
+ case "progress": return showProgress(data);
+ case "downloaded": return void reportPersistence();
+ case "loaded": return log(`Loaded ${data.model}: copy into MEMFS ${ms(data.timings.copyToMemfsMs)}, create recognizer ${ms(data.timings.createRecognizerMs)}, WebAssembly memory ${data.heapAfterLoad} MiB.`);
+ case "result": return log(`${data.label}: ${ms(data.ms)} for ${ms(data.audioMs)} of audio (RTF ${data.rtf.toFixed(2)}), memory ${data.heap} MiB\n → ${data.text}`);
+ case "utterance": return log(`[${ms(data.offsetMs)}] ${data.text}\n (${ms(data.ms)} for ${ms(data.audioMs)}, RTF ${data.rtf.toFixed(2)}, lag ${ms(data.lagMs)})`);
+ case "live-started": return log("Listening.");
+ case "live-stopped": return log("Stopped listening.");
+ case "deleted": return log(`Deleted ${data.model} from OPFS.`);
+ case "log": return console.debug(data.text);
+ case "error": return log(`Error: ${data.message}`);
+ }
+};
+worker.onerror = (e) => log(`Worker error: ${e.message}`);
+
+$("download").onclick = () => worker.postMessage({ type: "download", model: selectedModel() });
+$("load").onclick = () => {
+ log("Loading…");
+ worker.postMessage({ type: "load", model: selectedModel() });
+};
+$("delete").onclick = () => worker.postMessage({ type: "delete", model: selectedModel() });
+$("transcribe").onclick = async () => {
+ const [ label, url ] = CLIPS.find(([ clipLabel ]) => clipLabel === $("clip").value) ?? CLIPS[0];
+ const samples = await decodeClip(url);
+ worker.postMessage({ type: "transcribe", label, samples }, [ samples.buffer ]);
+};
+$("listen").onclick = async () => {
+ if (microphone) {
+ microphone.stop();
+ microphone = null;
+ worker.postMessage({ type: "live-stop" });
+ $("listen").textContent = "Listen to the microphone";
+ return;
+ }
+ worker.postMessage({ type: "live-start" });
+ microphone = await captureMicrophone((samples) =>
+ worker.postMessage({ type: "samples", samples }, [ samples.buffer ]));
+ $("listen").textContent = "Stop listening";
+};
+
+/** Asks the browser not to evict the models under storage pressure. Only the page can ask. */
+async function reportPersistence() {
+ const persisted = await navigator.storage.persist();
+ const { usage, quota } = await navigator.storage.estimate();
+ log(`Downloaded. Persistent storage: ${persisted}, using ${(usage / 1048576).toFixed(0)} of ${(quota / 1048576).toFixed(0)} MiB.`);
+}
+
+function selectedModel() {
+ const model = MODELS.find(({ id }) => id === $("model").value) ?? MODELS[0];
+ return {
+ id: model.id,
+ files: model.files.map(([ name, size ]) => ({ name, size, url: `${HF}/${model.repo}/resolve/${model.revision}/${name}` })),
+ modelConfig: model.modelConfig(`/models/${model.id}`)
+ };
+}
+
+/** Decodes and resamples a clip to mono 16 kHz with the browser's own decoder. */
+async function decodeClip(url) {
+ const bytes = await (await fetch(url)).arrayBuffer();
+ const context = new AudioContext({ sampleRate: 16000 });
+ try {
+ return (await context.decodeAudioData(bytes)).getChannelData(0).slice();
+ } finally {
+ await context.close();
+ }
+}
+
+async function captureMicrophone(onSamples) {
+ const media = await navigator.mediaDevices.getUserMedia({ audio: { channelCount: 1 } });
+ const context = new AudioContext({ sampleRate: 16000 });
+ const processor = context.createScriptProcessor(4096, 1, 1);
+ processor.onaudioprocess = (e) => onSamples(e.inputBuffer.getChannelData(0).slice());
+ context.createMediaStreamSource(media).connect(processor);
+ processor.connect(context.destination);
+ return {
+ stop() {
+ for (const track of media.getTracks()) {
+ track.stop();
+ }
+ void context.close();
+ }
+ };
+}
+
+function showProgress({ file, loaded, total, cached, ms: took }) {
+ const percent = Math.round(loaded / total * 100);
+ $("progress").textContent = cached
+ ? `${file}: already in OPFS`
+ : `${file}: ${(loaded / 1048576).toFixed(0)} / ${(total / 1048576).toFixed(0)} MiB (${percent}%)`;
+ if (took) {
+ log(`${file}: ${(total / 1048576).toFixed(0)} MiB in ${ms(took)}`);
+ }
+}
+
+function environment() {
+ return `crossOriginIsolated=${self.crossOriginIsolated}, cores=${navigator.hardwareConcurrency}, ${navigator.userAgent}`;
+}
+
+function ms(value) {
+ return value >= 1000 ? `${(value / 1000).toFixed(1)} s` : `${Math.round(value)} ms`;
+}
+
+function log(text) {
+ $("log").textContent += `${new Date().toLocaleTimeString()} ${text}\n`;
+ console.log(text);
+}
diff --git a/apps/standalone/transcription-poc/serve.mjs b/apps/standalone/transcription-poc/serve.mjs
new file mode 100644
index 0000000000..8cda7894c9
--- /dev/null
+++ b/apps/standalone/transcription-poc/serve.mjs
@@ -0,0 +1,39 @@
+// Serves the prototype on http://localhost:5555, on an origin of its own so that standalone's
+// service worker never sits between the page and a model download.
+import { createReadStream } from "node:fs";
+import { stat } from "node:fs/promises";
+import { createServer } from "node:http";
+import { extname, join, normalize } from "node:path";
+
+const PORT = 5555;
+const ROOT = import.meta.dirname;
+const TYPES = {
+ ".html": "text/html; charset=utf-8",
+ ".js": "text/javascript; charset=utf-8",
+ ".css": "text/css; charset=utf-8",
+ ".wasm": "application/wasm",
+ ".onnx": "application/octet-stream"
+};
+
+createServer(async (request, response) => {
+ const pathname = new URL(request.url ?? "/", "http://localhost").pathname;
+ const path = normalize(join(ROOT, pathname === "/" ? "index.html" : pathname));
+ if (!path.startsWith(ROOT)) {
+ response.writeHead(403).end();
+ return;
+ }
+
+ try {
+ const info = await stat(path);
+ response.writeHead(200, {
+ "Content-Type": TYPES[extname(path)] ?? "application/octet-stream",
+ "Content-Length": info.size,
+ // Matches standalone, which is not cross-origin isolated either.
+ "Cross-Origin-Opener-Policy": "same-origin",
+ "Cache-Control": "no-store"
+ });
+ createReadStream(path).pipe(response);
+ } catch {
+ response.writeHead(404).end();
+ }
+}).listen(PORT, () => console.log(`http://localhost:${PORT}/`));
diff --git a/apps/standalone/transcription-poc/worker.js b/apps/standalone/transcription-poc/worker.js
new file mode 100644
index 0000000000..86854944ec
--- /dev/null
+++ b/apps/standalone/transcription-poc/worker.js
@@ -0,0 +1,244 @@
+// Runs sherpa-onnx (WebAssembly) off the page's thread: downloads models into OPFS, loads them into
+// the Emscripten file system, and transcribes clips or live microphone audio.
+// A classic worker, because the prebuilt glue declares a global `Module` and is loaded with
+// importScripts().
+/* global Module, OfflineRecognizer, createVad */
+
+const SAMPLE_RATE = 16000;
+const VAD_WINDOW = 512;
+
+self.Module = {
+ locateFile: (file) => `vendor/${file}`,
+ // The baked-in model package is patched out by fetch-assets.mjs; answer its request with nothing.
+ getPreloadedPackage: () => new ArrayBuffer(0),
+ print: (text) => post("log", { text }),
+ printErr: (text) => post("log", { text }),
+ onRuntimeInitialized: () => post("ready", { heap: heapMiB() })
+};
+importScripts("vendor/sherpa-onnx-wasm-main-vad-asr.js", "vendor/sherpa-onnx-asr.js", "vendor/sherpa-onnx-vad.js");
+
+let recognizer = null;
+let vad = null;
+let pending = new Float32Array(0);
+let liveSamples = 0;
+
+self.onmessage = async ({ data }) => {
+ try {
+ switch (data.type) {
+ case "download": return await download(data.model);
+ case "load": return await load(data.model);
+ case "transcribe": return transcribeClip(data.label, data.samples);
+ case "live-start": return startLive();
+ case "samples": return acceptLive(data.samples);
+ case "live-stop": return stopLive();
+ case "delete": return await deleteModel(data.model);
+ }
+ } catch (e) {
+ post("error", { message: e?.stack ?? String(e) });
+ }
+};
+
+async function download(model) {
+ const dir = await modelDirectory(model, true);
+ for (const file of model.files) {
+ const handle = await dir.getFileHandle(file.name, { create: true });
+ if ((await handle.getFile()).size === file.size) {
+ post("progress", { file: file.name, loaded: file.size, total: file.size, cached: true });
+ continue;
+ }
+
+ const started = performance.now();
+ const response = await fetch(file.url);
+ if (!response.ok || !response.body) {
+ throw new Error(`${file.url}: HTTP ${response.status}`);
+ }
+ let loaded = 0;
+ let lastReport = 0;
+ const counter = new TransformStream({
+ transform(chunk, controller) {
+ loaded += chunk.byteLength;
+ if (performance.now() - lastReport > 250) {
+ lastReport = performance.now();
+ post("progress", { file: file.name, loaded, total: file.size });
+ }
+ controller.enqueue(chunk);
+ }
+ });
+ await writeStream(handle, response.body.pipeThrough(counter));
+ post("progress", { file: file.name, loaded, total: file.size, ms: performance.now() - started });
+ }
+ post("downloaded", { model: model.id });
+}
+
+/** Streams into an OPFS file, through the async writable where the browser has one (not Safari). */
+async function writeStream(handle, stream) {
+ if ("createWritable" in handle) {
+ await stream.pipeTo(await handle.createWritable());
+ return;
+ }
+ const access = await handle.createSyncAccessHandle();
+ try {
+ access.truncate(0);
+ let offset = 0;
+ for await (const chunk of stream) {
+ offset += access.write(chunk, { at: offset });
+ }
+ access.flush();
+ } finally {
+ access.close();
+ }
+}
+
+async function load(model) {
+ unload();
+ const timings = {};
+ const dir = await modelDirectory(model, false);
+ Module.FS_createPath("/", `models/${model.id}`, true, true);
+
+ let started = performance.now();
+ for (const file of model.files) {
+ const blob = await (await dir.getFileHandle(file.name)).getFile();
+ // canOwn: the file system adopts the buffer instead of copying it.
+ Module.FS_createDataFile(`/models/${model.id}`, file.name, new Uint8Array(await blob.arrayBuffer()), true, true, true);
+ }
+ const vadBytes = new Uint8Array(await (await fetch("vendor/silero_vad.onnx")).arrayBuffer());
+ try {
+ Module.FS_unlink("/models/silero_vad.onnx");
+ } catch {
+ // First load.
+ }
+ Module.FS_createDataFile("/models", "silero_vad.onnx", vadBytes, true, true, true);
+ timings.copyToMemfsMs = performance.now() - started;
+
+ started = performance.now();
+ recognizer = new OfflineRecognizer({
+ featConfig: { sampleRate: SAMPLE_RATE, featureDim: 80 },
+ modelConfig: { ...model.modelConfig, numThreads: 1, provider: "cpu", debug: 0 },
+ decodingMethod: "greedy_search"
+ }, Module);
+ timings.createRecognizerMs = performance.now() - started;
+ const heapAfterLoad = heapMiB();
+
+ // The recognizer holds its own copy now, so the file system's can go.
+ for (const file of model.files) {
+ Module.FS_unlink(`/models/${model.id}/${file.name}`);
+ }
+ post("loaded", { model: model.id, timings, heapAfterLoad, heap: heapMiB() });
+}
+
+function unload() {
+ recognizer?.free();
+ recognizer = null;
+ vad?.free();
+ vad = null;
+}
+
+function transcribeClip(label, samples) {
+ requireRecognizer();
+ const started = performance.now();
+ const text = decode(samples);
+ const ms = performance.now() - started;
+ const audioMs = samples.length / SAMPLE_RATE * 1000;
+ post("result", { label, text, ms, audioMs, rtf: ms / audioMs, heap: heapMiB() });
+}
+
+function startLive() {
+ requireRecognizer();
+ vad?.free();
+ vad = createVad(Module, {
+ sileroVad: {
+ model: "/models/silero_vad.onnx",
+ threshold: 0.5,
+ minSilenceDuration: 0.5,
+ minSpeechDuration: 0.25,
+ maxSpeechDuration: 8,
+ windowSize: VAD_WINDOW
+ },
+ tenVad: { model: "", threshold: 0.5, minSilenceDuration: 0.5, minSpeechDuration: 0.25, maxSpeechDuration: 20, windowSize: 256 },
+ sampleRate: SAMPLE_RATE,
+ numThreads: 1,
+ provider: "cpu",
+ debug: 0,
+ bufferSizeInSeconds: 60
+ });
+ pending = new Float32Array(0);
+ liveSamples = 0;
+ post("live-started", {});
+}
+
+function acceptLive(samples) {
+ if (!vad) {
+ return;
+ }
+ liveSamples += samples.length;
+ const all = new Float32Array(pending.length + samples.length);
+ all.set(pending);
+ all.set(samples, pending.length);
+ let offset = 0;
+ for (; offset + VAD_WINDOW <= all.length; offset += VAD_WINDOW) {
+ vad.acceptWaveform(all.subarray(offset, offset + VAD_WINDOW));
+ }
+ pending = all.slice(offset);
+ drainVad();
+}
+
+function stopLive() {
+ if (!vad) {
+ return;
+ }
+ vad.flush();
+ drainVad();
+ vad.free();
+ vad = null;
+ post("live-stopped", {});
+}
+
+function drainVad() {
+ while (!vad.isEmpty()) {
+ const segment = vad.front();
+ vad.pop();
+ const started = performance.now();
+ const text = decode(segment.samples);
+ const ms = performance.now() - started;
+ const audioMs = segment.samples.length / SAMPLE_RATE * 1000;
+ // How far the transcript trails the speaker once this utterance is out.
+ const lagMs = (liveSamples - segment.start) / SAMPLE_RATE * 1000 - audioMs + ms;
+ post("utterance", { text, ms, audioMs, rtf: ms / audioMs, lagMs, offsetMs: segment.start / SAMPLE_RATE * 1000 });
+ }
+}
+
+function decode(samples) {
+ const stream = recognizer.createStream();
+ try {
+ stream.acceptWaveform(SAMPLE_RATE, samples);
+ recognizer.decode(stream);
+ return recognizer.getResult(stream).text.trim();
+ } finally {
+ stream.free();
+ }
+}
+
+async function deleteModel(model) {
+ const root = await navigator.storage.getDirectory();
+ await root.removeEntry(model.id, { recursive: true }).catch(() => {});
+ post("deleted", { model: model.id });
+}
+
+async function modelDirectory(model, create) {
+ const root = await navigator.storage.getDirectory();
+ return await root.getDirectoryHandle(model.id, { create });
+}
+
+function requireRecognizer() {
+ if (!recognizer) {
+ throw new Error("Load a model first.");
+ }
+}
+
+function heapMiB() {
+ return Math.round(Module.HEAP8.length / 1024 / 1024);
+}
+
+function post(type, data) {
+ self.postMessage({ type, ...data });
+}