Record your screen, ship a demo. Free and open-source, GPU-accelerated, no watermarks, no subscriptions. Windows, macOS, Linux. Actively maintained.
Word-timing harness
Scores transcript word times against exact ground truth: how far each word boundary is from the audio, and what deleting one word or one phrase in the transcript editor would leave audible or clip. Method, baseline and plan: issue #948. How the app times words: transcription-and-captions.md § Word-level alignment.
Node 22+, no dependencies. The corpus is synthesized with Windows speech synthesis, so generating it needs Windows; scoring runs anywhere.
Data
Everything generated goes to data/ next to these scripts (gitignored), or
wherever OSC_WORD_TIMING_DATA points:
clips/:<id>.wav(clean),<id>.noisy.wav,<id>.ref.json(reference words)corpus-manifest.jsonresults/raw/<tag>/: the helper's raw responses;results/<name>.{txt,json}: scores
Generate the corpus (Windows, once, about 5 min)
node make-corpus.mjs # 48 TTS clips + a noisy copy of each + reference times
node validate-ref.mjs # checks the reference against the audio's energy
- Voices: OneCore Hortense, Paul and Julie (French), SAPI Zira (English).
Scripts are in
corpus-texts.mjs, synthesis intts.ps1. - ffmpeg:
electron/native/bin/win32-x64/ffmpeg.exe, or setFFMPEG.
Run and score
node run-helper.mjs <tag> <path/to/whisper-stt-server.exe> [--cpu]
node run-align.mjs <tag> <tag>-ctc <path/to/whisper-stt-server.exe> \
--model en=<w2v-en-base-q8_0.gguf> --model fr=<w2v-fr-large-q8_0.gguf> [--cpu]
node evaluate.mjs <tag>-ctc [--snap <post-pass.ts>] [--ctc <ctcAlign.ts>] [--out <name>]
node summarize.mjs "Before=<baseline>" "After=<name>"
run-helper.mjsstarts its own helper on a port in 20500-20599 (or the 100 ports fromOSC_WORD_TIMING_PORT, to run beside another agent) with the app's models (%APPDATA%/openscreen/stt-models/whisper-ggml:ggml-small-q8_0.binandggml-silero-v6.2.0.bin), sends every clip like the app does, and stops it. Vulkan takes about 2 min for the 55 min of audio, CPU about 17.evaluate.mjsparses the responses aswhisperServer.tsdoes and runs the post-pass: the repo'selectron/stt/snapWordBoundaries.tsby default, or the file given with--snap. It reports three stages:raw,post(no speech intervals) andpost+vad(what the app ships without an aligner).run-align.mjsadds the CTC aligner's letter scores (/emissions) to saved responses, without running whisper again, for the language each clip was detected in. On those,evaluate.mjsaddsctc(the aligner,electron/stt/ctcAlign.tsor--ctc) andctc+vad(what the app ships with an aligner), and the aligner's share of the runtime.- The aligner GGUFs come from
scripts/convert-wav2vec2-gguf.mjs(sources in its header), or from the app's cache:stt-models/ctc-aligner/. - A baseline is the same two commands on an older helper and post-pass:
build the helper at that revision, and pass
git show <rev>:electron/stt/snapWordBoundaries.tssaved to a file as--snap.
Real speech
LibriSpeech, against forced-aligned word times (English read speech; the Montreal Forced Aligner reference is itself 10 to 20 ms from a human's):
# test-clean.tar.gz from https://www.openslr.org/12, librispeech_alignments.zip
# from https://zenodo.org/records/2619474 (both CC-BY-4.0, dev-time only)
export OSC_WORD_TIMING_DATA=<another data dir>
node make-librispeech.mjs <LibriSpeech/test-clean> <alignments/test-clean> # 2 clips x 40 speakers, ~46 min
node run-helper.mjs ls <exe> --condition clean
node run-align.mjs ls ls-ctc <exe> --model en=<w2v-en-base-q8_0.gguf>
node evaluate.mjs ls-ctc
A take of your own, with no ground truth:
node real-check.mjs <whisper-stt-server.exe> take.wav [--align <aligner.gguf>] # 16 kHz mono s16
It prints how far each word's first-token time lies after its start, and where
each phrase's first word lands against the VAD onset, raw and after the
post-pass. With --align, every word is listed with its DTW and aligner
times, so the boundaries that moved can be checked by ear or on a spectrogram.
Reading the numbers
- Positions come from the reference: phrase-initial has a pause of at least 100 ms before it, phrase-final after it, inner is everything else.
- A delete is clean when at most 20 ms of the deleted word stays audible and at most 20 ms of its neighbours is cut.
- Synthetic speech flatters every method. Use the harness to rank approaches, and check a winner on real speech.
- The aligner's calibration (
START_OFFSET_SECand the others inctcAlign.ts) was fitted on the TTS corpus and checked on LibriSpeech; refit both before changing a model.