Skip to content

Desktop app that records your on-screen work session and uses the GitHub Copilot SDK to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Autopilots like Scout, Microsoft Copilot Cowork, or Copilot Studio.

Support multilingual narration transcription

Giorgio Ughini99e750058474
diff --git a/README.md b/README.md
index 9c542e9..4395a2b 100644
--- a/README.md
+++ b/README.md
@@ -44,8 +44,12 @@ activity" panel shows exactly what's collected:
- **Narration** *(optional)* — turn on **Narrate** before capture or toggle the
microphone from the floating recording bar. Each microphone-on interval is
timestamped on the video timeline, saved locally, and can be transcribed
- **on-device** (Whisper via transformers.js). The first transcription uses an
- explicit, one-time ~250 MB model download.
+ **on-device** in English, Italian, French, or Spanish (Whisper via
+ transformers.js), preserving the selected language. The first transcription
+ uses an explicit, one-time ~252 MB model download. The multilingual q8
+ checkpoint is only about 0.6 MB larger than the previous English-only model
+ and uses the same `small` architecture, so runtime memory and transcription
+ speed are expected to remain effectively unchanged.
## Requirements
diff --git a/THIRD-PARTY-NOTICES.md b/THIRD-PARTY-NOTICES.md
index 15bec55..fb1c7f1 100644
--- a/THIRD-PARTY-NOTICES.md
+++ b/THIRD-PARTY-NOTICES.md
@@ -11,6 +11,22 @@ the other native runtime components.
The dependency tree is otherwise permissive (MIT, ISC, Apache-2.0, BSD,
BlueOak-1.0.0) and compatible with distributing this application under MIT.
+## Optional downloaded model
+
+### OpenAI Whisper small — `Xenova/whisper-small`
+- The multilingual model is downloaded from
+ [`Xenova/whisper-small`](https://huggingface.co/Xenova/whisper-small) only
+ after explicit user approval; its weights are not bundled with Skill Recorder.
+- The Transformers.js-compatible ONNX conversion is published by Xenova
+ (Joshua Lochner) from OpenAI's
+ [`openai/whisper-small`](https://huggingface.co/openai/whisper-small)
+ checkpoint.
+- The Hugging Face model metadata declares **Apache-2.0**. OpenAI's
+ [Whisper repository](https://github.com/openai/whisper) also states that its
+ code and model weights are released under the **MIT License**. The downloaded
+ model remains subject to its publisher's applicable terms and does not change
+ Skill Recorder's MIT license.
+
## Bundled runtime components
### GitHub Copilot CLI — `@github/copilot` (+ platform binary `@github/copilot-<platform>-<arch>`)
diff --git a/common/audio.test.ts b/common/audio.test.ts
index 8091ec7..353dd6b 100644
--- a/common/audio.test.ts
+++ b/common/audio.test.ts
@@ -5,6 +5,7 @@ import {
AUDIO_MANIFEST_VERSION,
alignAudioSegmentsToVideo,
isAudioManifestV2,
+ readAudioNarrationLanguage,
readAudioSources,
type AudioSegment,
} from "./audio";
@@ -80,3 +81,38 @@ test("legacy and segmented metadata normalize to timestamped transcription sourc
);
assert.throws(() => readAudioSources({ version: AUDIO_MANIFEST_VERSION, segments: [{}] }));
});
+
+test("audio metadata preserves a supported language and defaults legacy recordings to English", () => {
+ assert.equal(
+ readAudioNarrationLanguage({
+ version: AUDIO_MANIFEST_VERSION,
+ narrationLanguage: "it",
+ segments: [],
+ }),
+ "it",
+ );
+ assert.equal(
+ readAudioNarrationLanguage({
+ version: AUDIO_MANIFEST_VERSION,
+ segments: [],
+ }),
+ "en",
+ );
+ assert.equal(
+ readAudioNarrationLanguage({
+ file: "audio.webm",
+ startEpoch: 1_100,
+ durationMs: 400,
+ }),
+ "en",
+ );
+ assert.throws(
+ () =>
+ readAudioNarrationLanguage({
+ version: AUDIO_MANIFEST_VERSION,
+ narrationLanguage: "de",
+ segments: [],
+ }),
+ /unsupported transcription language/i,
+ );
+});
diff --git a/common/audio.ts b/common/audio.ts
index 3edb0bc..e6771ad 100644
--- a/common/audio.ts
+++ b/common/audio.ts
@@ -1,3 +1,9 @@
+import {
+ DEFAULT_NARRATION_LANGUAGE,
+ isNarrationLanguage,
+ type NarrationLanguage,
+} from "./narration";
+
/** Version written by new recordings that can toggle the microphone mid-session. */
export const AUDIO_MANIFEST_VERSION = 2 as const;
@@ -18,6 +24,8 @@ export interface AudioSegment {
export interface AudioManifestV2 {
version: typeof AUDIO_MANIFEST_VERSION;
+ /** Source language selected before recording. Absent in older v2 manifests. */
+ narrationLanguage?: NarrationLanguage;
segments: AudioSegment[];
}
@@ -69,6 +77,17 @@ export function readAudioSources(value: unknown): AudioSource[] {
return normalized;
}
+/** Read the session's transcription language, preserving English for older recordings. */
+export function readAudioNarrationLanguage(value: unknown): NarrationLanguage {
+ if (!isAudioManifestV2(value) || value.narrationLanguage === undefined) {
+ return DEFAULT_NARRATION_LANGUAGE;
+ }
+ if (!isNarrationLanguage(value.narrationLanguage)) {
+ throw new Error("Narration metadata contains an unsupported transcription language.");
+ }
+ return value.narrationLanguage;
+}
+
/** Add video-relative offsets without altering the session-relative timeline. */
export function alignAudioSegmentsToVideo(
segments: readonly AudioSegment[],
diff --git a/common/ipc.ts b/common/ipc.ts
index fca66e5..62ac20d 100644
--- a/common/ipc.ts
+++ b/common/ipc.ts
@@ -1,5 +1,6 @@
import type { Analysis, AnalysisFeedback, AnalysisStep, Confidence } from "./analysis";
import type { AutomationPlan, BuiltAutomation } from "./automation";
+import type { NarrationLanguage } from "./narration";
import type { BuiltSkill, SkillArchitecture, SkillPlan } from "./skill";
import type { RecorderState } from "./types";
@@ -23,6 +24,8 @@ export interface SessionSummary {
hasVideo: boolean;
/** True when the user opted into narration and usable audio was saved. */
hasAudio: boolean;
+ /** Selected source language for saved audio, or null when no audio exists. */
+ narrationLanguage: NarrationLanguage | null;
/** True once transcription completed, including recordings with no detected speech. */
hasNarration: boolean;
narrationSegmentCount: number | null;
@@ -67,6 +70,8 @@ export interface RecorderStatus {
state: RecorderState;
sessionId: string | null;
startedAt: number | null;
+ /** Source language fixed for the active recording's narration. */
+ narrationLanguage: NarrationLanguage;
eventCount: number;
transition: "none" | "starting" | "stopping" | "discarding";
microphone: {
@@ -190,6 +195,8 @@ export interface StartResult {
export interface StartOptions {
/** Capture microphone narration for this session (opt-in, off by default). */
narration?: boolean;
+ /** Source language to preserve in the transcript. Defaults to English. */
+ narrationLanguage?: NarrationLanguage;
}
export interface StopResult {
@@ -211,6 +218,12 @@ export interface MicrophoneResult {
error?: string;
}
+export interface NarrationLanguageResult {
+ ok: boolean;
+ language?: NarrationLanguage;
+ error?: string;
+}
+
export interface MarkerResult {
ok: boolean;
error?: string;
@@ -267,6 +280,7 @@ export const IPC = {
stop: "recorder:stop",
discard: "recorder:discard",
microphone: "recorder:microphone",
+ narrationLanguage: "recorder:narration-language",
status: "recorder:status",
marker: "recorder:marker",
doctor: "doctor:check",
@@ -306,6 +320,7 @@ export interface SkillRecorderApi {
stop(): Promise<StopResult>;
discard(): Promise<DiscardResult>;
setMicrophoneEnabled(enabled: boolean): Promise<MicrophoneResult>;
+ setNarrationLanguage(language: NarrationLanguage): Promise<NarrationLanguageResult>;
status(): Promise<RecorderStatus>;
marker(note: string): Promise<MarkerResult>;
doctor(): Promise<DoctorReport>;
diff --git a/common/narration.ts b/common/narration.ts
index e0a20af..d084d4a 100644
--- a/common/narration.ts
+++ b/common/narration.ts
@@ -1,3 +1,28 @@
+export const NARRATION_LANGUAGE_CODES = ["en", "it", "fr", "es"] as const;
+export type NarrationLanguage = (typeof NARRATION_LANGUAGE_CODES)[number];
+
+export const DEFAULT_NARRATION_LANGUAGE: NarrationLanguage = "en";
+export const NARRATION_LANGUAGE_LABELS = {
+ en: "English",
+ it: "Italian",
+ fr: "French",
+ es: "Spanish",
+} satisfies Record<NarrationLanguage, string>;
+
+/** Current q8 model files total 251,875,316 bytes at the publisher. */
+export const NARRATION_MODEL_DOWNLOAD_LABEL = "~252 MB";
+
+export function isNarrationLanguage(value: unknown): value is NarrationLanguage {
+ return (
+ typeof value === "string" &&
+ (NARRATION_LANGUAGE_CODES as readonly string[]).includes(value)
+ );
+}
+
+export function narrationLanguageLabel(language: NarrationLanguage): string {
+ return NARRATION_LANGUAGE_LABELS[language];
+}
+
/**
* The offline voice-narration transcript. This is the spoken form of the typed
* marker: the user's own stated intent while recording. It is produced after
@@ -18,6 +43,8 @@ export interface NarrationSegment {
export interface NarrationTranscript {
/** The Whisper model id that produced this transcript. */
model: string;
+ /** The explicitly selected source language, preserved rather than translated. */
+ language: NarrationLanguage;
segments: NarrationSegment[];
}
diff --git a/docs/windows-capture.md b/docs/windows-capture.md
index 0633cb2..30a415d 100644
--- a/docs/windows-capture.md
+++ b/docs/windows-capture.md
@@ -60,15 +60,18 @@ Run a real recording on Windows and verify each source lands in the session's
5. **Video.** Confirm `video.webm`, `video-frames.json`, snapshots under
`video-frames/`, and retained images under `frames/`.
6. **Voice narration.** Turn on the **Narrate** switch before starting or use the
- microphone control in the floating recording bar. Toggle it off and on around
- two spoken sentences. After Stop, confirm `audio.json` contains a version-2
- segment manifest and the referenced `audio/segment-*.webm` files exist. Their
- session/video offsets should preserve the silent gap while the microphone was
- off. If the model is not installed, use **Download &
- transcribe** in Sessions (or the actionable HUD readiness row) to approve the
- one-time ~250 MB Whisper download. Confirm `narration.json` then contains your
- words with `atMs` offsets. Later runs are offline. On Windows the mic grant is
- requested by the OS on first use.
+ microphone control in the floating recording bar. Select English, Italian,
+ French, or Spanish before recording, then toggle the microphone off and on
+ around two spoken sentences. After Stop, confirm `audio.json` contains a
+ version-2 segment manifest with `narrationLanguage` and that the referenced
+ `audio/segment-*.webm` files exist. Their session/video offsets should
+ preserve the silent gap while the microphone was off. If the model is not
+ installed, use **Download & transcribe** in Sessions (or the actionable HUD
+ readiness row) to approve the one-time ~252 MB Whisper download. Confirm
+ `narration.json` records the chosen language and contains the
+ original-language words with `atMs` offsets. Repeat with all four supported
+ languages. Later runs are offline. On Windows the mic grant is requested by
+ the OS on first use.
7. **Recording controls.** Confirm the floating bar stays above the active app,
can be dragged without making its buttons unclickable, and does not appear in
captured frames where Windows capture protection is supported. Canceling the
@@ -89,8 +92,12 @@ modules (`koffi`, `@koromix/*`, `sharp`, `@img/*`,
are listed under `asarUnpack` so
their binaries load from disk rather than from inside the asar archive. The
Whisper model itself is not bundled. It downloads to the app's user-data
-`models` folder only after the user approves the one-time ~250 MB download from
+`models` folder only after the user approves the one-time ~252 MB download from
the HUD or Sessions; recording and core session processing do not wait for it.
+The multilingual q8 files total about 251.9 MB versus 251.2 MB for the previous
+English-only checkpoint. Both use the same Whisper `small` architecture, so
+runtime memory and transcription speed are expected to remain effectively
+unchanged.
Build each Windows installer on its matching native machine or CI runner so npm
selects the correct optional packages:
diff --git a/electron/audio/recorder.ts b/electron/audio/recorder.ts
index fa19bf9..49114ce 100644
--- a/electron/audio/recorder.ts
+++ b/electron/audio/recorder.ts
@@ -12,6 +12,7 @@ import {
type AudioSegment,
type LegacyAudioMetadata,
} from "../../common/audio";
+import type { NarrationLanguage } from "../../common/narration";
import { createLogger } from "../logger";
import type { AudioCaptureEnded } from "../recorder/controller";
@@ -54,6 +55,7 @@ export class AudioRecorder {
private win: BrowserWindow | null = null;
private dir = "";
private sessionStartedAt = 0;
+ private narrationLanguage: NarrationLanguage = "en";
private sequence = 0;
private active: PendingSegment | null = null;
private disableTask: Promise<AudioSegment | null> | null = null;
@@ -111,10 +113,15 @@ export class AudioRecorder {
};
/** Prepare the session's hidden audio utility window without touching the mic. */
- async start(sessionDir: string, sessionStartedAt: number): Promise<void> {
+ async start(
+ sessionDir: string,
+ sessionStartedAt: number,
+ narrationLanguage: NarrationLanguage,
+ ): Promise<void> {
if (this.win) throw new Error("Microphone recorder is already initialized.");
this.dir = sessionDir;
this.sessionStartedAt = sessionStartedAt;
+ this.narrationLanguage = narrationLanguage;
this.sequence = 0;
this.segments.length = 0;
await mkdir(path.join(this.dir, AUDIO_DIR), { recursive: true });
@@ -287,7 +294,13 @@ export class AudioRecorder {
const segments = alignAudioSegmentsToVideo(this.segments, videoStartEpoch);
const manifest: AudioManifestV2 | null =
- segments.length > 0 ? { version: AUDIO_MANIFEST_VERSION, segments } : null;
+ segments.length > 0
+ ? {
+ version: AUDIO_MANIFEST_VERSION,
+ narrationLanguage: this.narrationLanguage,
+ segments,
+ }
+ : null;
try {
if (manifest) {
diff --git a/electron/ipc.ts b/electron/ipc.ts
index fb629ee..b515ca4 100644
--- a/electron/ipc.ts
+++ b/electron/ipc.ts
@@ -16,6 +16,7 @@ import type {
} from "../common/ipc";
import { IPC } from "../common/ipc";
import type { AutomationPlan } from "../common/automation";
+import type { NarrationLanguage } from "../common/narration";
import type { SkillPlan } from "../common/skill";
import { AutomationBuilder, loadPersistedAutomation } from "./automationbuilder/builder";
import { Describer, loadPersistedAnalysis } from "./describer/describer";
@@ -43,6 +44,9 @@ export function registerIpc(
ipcMain.handle(IPC.microphone, (_event, enabled: boolean) =>
recorder.setMicrophoneEnabled(enabled),
);
+ ipcMain.handle(IPC.narrationLanguage, (_event, language: NarrationLanguage) =>
+ recorder.setNarrationLanguage(language),
+ );
ipcMain.handle(IPC.status, () => recorder.status());
ipcMain.handle(IPC.marker, (_event, note: string) => recorder.marker(note));
ipcMain.handle(IPC.doctor, () => runDoctor());
diff --git a/electron/narration/manager.ts b/electron/narration/manager.ts
index d352a48..7335eac 100644
--- a/electron/narration/manager.ts
+++ b/electron/narration/manager.ts
@@ -3,6 +3,7 @@ import { readdir, rename, rm, writeFile } from "node:fs/promises";
import path from "node:path";
import {
+ readAudioNarrationLanguage,
readAudioSources,
type AudioSource,
} from "../../common/audio";
@@ -10,7 +11,13 @@ import type {
NarrationActionResult,
NarrationStatus,
} from "../../common/ipc";
-import { NARRATION_FILE, type NarrationTranscript } from "../../common/narration";
+import {
+ DEFAULT_NARRATION_LANGUAGE,
+ isNarrationLanguage,
+ NARRATION_FILE,
+ type NarrationLanguage,
+ type NarrationTranscript,
+} from "../../common/narration";
import type { SessionMeta } from "../../common/types";
import { createLogger } from "../logger";
import { isValidSessionId, sessionDir } from "../recorder/session-store";
@@ -19,6 +26,7 @@ import {
getAsrPipeline,
isNarrationModelCached,
narrationModelCacheDir,
+ removeLegacyNarrationModelCache,
type ModelLoadProgress,
} from "./whisper";
@@ -36,8 +44,11 @@ function readTranscript(file: string): NarrationTranscript | null {
if (!existsSync(file)) return null;
try {
const transcript = readJson<NarrationTranscript>(file);
- return typeof transcript.model === "string" && Array.isArray(transcript.segments)
- ? transcript
+ const language = transcript.language ?? DEFAULT_NARRATION_LANGUAGE;
+ return typeof transcript.model === "string" &&
+ isNarrationLanguage(language) &&
+ Array.isArray(transcript.segments)
+ ? { ...transcript, language }
: null;
} catch {
return null;
@@ -100,11 +111,13 @@ export class NarrationManager {
constructor(private readonly emitStatus: (status: NarrationStatus) => void) {}
initialize(): void {
+ const ready = isNarrationModelCached();
this.update({
- model: isNarrationModelCached() ? "ready" : "missing",
+ model: ready ? "ready" : "missing",
phase: "idle",
error: null,
});
+ if (ready) void this.cleanupLegacyModel();
}
status(): NarrationStatus {
@@ -205,9 +218,12 @@ export class NarrationManager {
let meta: SessionMeta;
let audioSources: AudioSource[];
+ let narrationLanguage: NarrationLanguage;
try {
meta = readJson<SessionMeta>(path.join(dir, "session.json"));
- audioSources = readAudioSources(readJson<unknown>(audioJson));
+ const audioMetadata = readJson<unknown>(audioJson);
+ audioSources = readAudioSources(audioMetadata);
+ narrationLanguage = readAudioNarrationLanguage(audioMetadata);
} catch (err) {
return { ok: false, error: `Could not read narration metadata: ${message(err)}` };
}
@@ -249,12 +265,17 @@ export class NarrationManager {
activeSessionId: sessionId,
error: null,
});
- const transcript: NarrationTranscript = { model: "", segments: [] };
+ const transcript: NarrationTranscript = {
+ model: "",
+ language: narrationLanguage,
+ segments: [],
+ };
for (const source of sourceFiles) {
const segmentTranscript = await transcribeNarration(
source.audioPath,
source.startEpoch - meta.startedAt,
source.durationMs,
+ narrationLanguage,
pipe,
);
transcript.model ||= segmentTranscript.model;
@@ -312,6 +333,7 @@ export class NarrationManager {
onProgress: (progress) =>
this.onModelProgress(progress, activeSessionId, downloading),
});
+ await this.cleanupLegacyModel();
this.update({
model: "ready",
phase: "idle",
@@ -324,6 +346,14 @@ export class NarrationManager {
return pipe;
}
+ private async cleanupLegacyModel(): Promise<void> {
+ try {
+ await removeLegacyNarrationModelCache();
+ } catch (error) {
+ log.warn("could not remove the superseded English narration model:", message(error));
+ }
+ }
+
private onModelProgress(
progress: ModelLoadProgress,
activeSessionId: string | null,
diff --git a/electron/narration/transcribe.test.ts b/electron/narration/transcribe.test.ts
new file mode 100644
index 0000000..9805667
--- /dev/null
+++ b/electron/narration/transcribe.test.ts
@@ -0,0 +1,29 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import {
+ isMeaningfulNarrationText,
+ narrationTranscriptionOptions,
+} from "./transcribe";
+
+test("multilingual transcription preserves the explicitly selected source language", () => {
+ assert.deepEqual(narrationTranscriptionOptions("fr"), {
+ return_timestamps: true,
+ chunk_length_s: 30,
+ stride_length_s: 5,
+ language: "fr",
+ task: "transcribe",
+ });
+});
+
+test("narration filtering accepts Unicode letters", () => {
+ assert.equal(isMeaningfulNarrationText("È già pronto"), true);
+ assert.equal(isMeaningfulNarrationText("日本語の説明"), true);
+ assert.equal(isMeaningfulNarrationText("123 ..."), false);
+});
+
+test("narration filtering removes localized Whisper boilerplate", () => {
+ assert.equal(isMeaningfulNarrationText("Merci d'avoir regardé !"), false);
+ assert.equal(isMeaningfulNarrationText("Gracias por ver."), false);
+ assert.equal(isMeaningfulNarrationText("Grazie per aver guardato"), false);
+});
diff --git a/electron/narration/transcribe.ts b/electron/narration/transcribe.ts
index b5c9bda..e871788 100644
--- a/electron/narration/transcribe.ts
+++ b/electron/narration/transcribe.ts
@@ -1,4 +1,4 @@
-import type { NarrationSegment } from "../../common/narration";
+import type { NarrationLanguage, NarrationSegment } from "../../common/narration";
import { decodeAudioFile } from "../audio/decode";
import { createLogger } from "../logger";
import { detectSilence, type SilenceSpan } from "./audio-analysis";
@@ -21,6 +21,7 @@ export async function transcribeNarration(
audioPath: string,
anchorDeltaMs: number,
durationMs: number,
+ language: NarrationLanguage,
pipe: AsrPipeline,
): Promise<{ model: string; segments: NarrationSegment[] }> {
const samples = await decodeAudioFile(audioPath);
@@ -30,11 +31,7 @@ export async function transcribeNarration(
const silences = detectSilence(samples);
- const result = await pipe(samples, {
- return_timestamps: true,
- chunk_length_s: 30,
- stride_length_s: 5,
- });
+ const result = await pipe(samples, narrationTranscriptionOptions(language));
const durationSec = durationMs > 0 ? durationMs / 1000 : samples.length / SAMPLE_RATE;
const raw = result.chunks?.length
@@ -44,7 +41,7 @@ export async function transcribeNarration(
const segments: NarrationSegment[] = [];
for (const chunk of raw) {
const text = (chunk.text ?? "").trim();
- if (!isMeaningful(text)) continue;
+ if (!isMeaningfulNarrationText(text)) continue;
const startSec = chunk.timestamp?.[0] ?? 0;
const endSec = chunk.timestamp?.[1] ?? Math.min(durationSec, startSec + 2);
@@ -74,19 +71,41 @@ const BOILERPLATE = new Set([
"you",
"thank you",
"thanks",
- "thank you.",
"thanks for watching",
- "thanks for watching!",
"please subscribe",
- "subtitles by the amara.org community",
+ "subtitles by the amara org community",
"bye",
- "bye.",
- ".",
+ "grazie",
+ "grazie per aver guardato",
+ "iscriviti al canale",
+ "ciao",
+ "merci",
+ "merci d avoir regardé",
+ "abonnez vous",
+ "au revoir",
+ "gracias",
+ "gracias por ver",
+ "suscríbete",
+ "adiós",
]);
-function isMeaningful(text: string): boolean {
+export function narrationTranscriptionOptions(language: NarrationLanguage) {
+ return {
+ return_timestamps: true,
+ chunk_length_s: 30,
+ stride_length_s: 5,
+ language,
+ task: "transcribe",
+ } as const;
+}
+
+export function isMeaningfulNarrationText(text: string): boolean {
if (text.length < 2) return false;
- if (!/[a-z]/i.test(text)) return false;
- const normalized = text.toLowerCase().replace(/\s+/g, " ").trim();
+ if (!/\p{L}/u.test(text)) return false;
+ const normalized = text
+ .normalize("NFKC")
+ .toLocaleLowerCase()
+ .replace(/[^\p{L}\p{N}]+/gu, " ")
+ .trim();
return !BOILERPLATE.has(normalized);
}
diff --git a/electron/narration/whisper.test.ts b/electron/narration/whisper.test.ts
new file mode 100644
index 0000000..ddeffeb
--- /dev/null
+++ b/electron/narration/whisper.test.ts
@@ -0,0 +1,52 @@
+import assert from "node:assert/strict";
+import { access, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import test from "node:test";
+
+import {
+ isNarrationModelCachedAt,
+ LEGACY_NARRATION_MODEL_ID,
+ NARRATION_MODEL_FILES,
+ NARRATION_MODEL_ID,
+ removeLegacyNarrationModelCache,
+} from "./whisper";
+
+async function seedModel(cacheDir: string, modelId: string): Promise<string> {
+ const modelDir = path.join(cacheDir, ...modelId.split("/"));
+ for (const relativeFile of NARRATION_MODEL_FILES) {
+ const file = path.join(modelDir, relativeFile);
+ await mkdir(path.dirname(file), { recursive: true });
+ await writeFile(file, "");
+ }
+ return modelDir;
+}
+
+test("cache detection selects the multilingual model rather than the legacy English model", async () => {
+ const root = await mkdtemp(path.join(tmpdir(), "skill-recorder-models-"));
+ try {
+ assert.equal(NARRATION_MODEL_ID, "Xenova/whisper-small");
+ await seedModel(root, LEGACY_NARRATION_MODEL_ID);
+ assert.equal(isNarrationModelCachedAt(root), false);
+
+ await seedModel(root, NARRATION_MODEL_ID);
+ assert.equal(isNarrationModelCachedAt(root), true);
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
+
+test("legacy model cleanup retains the multilingual cache", async () => {
+ const root = await mkdtemp(path.join(tmpdir(), "skill-recorder-models-"));
+ try {
+ const legacyDir = await seedModel(root, LEGACY_NARRATION_MODEL_ID);
+ await seedModel(root, NARRATION_MODEL_ID);
+
+ await removeLegacyNarrationModelCache(root);
+
+ assert.equal(isNarrationModelCachedAt(root), true);
+ await assert.rejects(access(legacyDir), { code: "ENOENT" });
+ } finally {
+ await rm(root, { recursive: true, force: true });
+ }
+});
diff --git a/electron/narration/whisper.ts b/electron/narration/whisper.ts
index 67bbac7..426e9bd 100644
--- a/electron/narration/whisper.ts
+++ b/electron/narration/whisper.ts
@@ -1,14 +1,15 @@
import { existsSync } from "node:fs";
+import { rm } from "node:fs/promises";
import path from "node:path";
import { app } from "electron";
-// Whisper small.en: English-only, int8 (q8) ONNX weights (~250 MB). Downloaded
-// once into the user-data models cache, then used fully offline. base.en is the
-// lighter fallback if this ever proves too heavy on low-end machines.
-const MODEL_ID = "Xenova/whisper-small.en";
+// Multilingual Whisper small, int8 (q8) ONNX weights (~252 MB). It has the same
+// architecture and runtime profile as the previous English-only checkpoint.
+export const NARRATION_MODEL_ID = "Xenova/whisper-small";
+export const LEGACY_NARRATION_MODEL_ID = "Xenova/whisper-small.en";
const DTYPE = "q8";
-const REQUIRED_MODEL_FILES = [
+export const NARRATION_MODEL_FILES = [
"config.json",
"generation_config.json",
"preprocessor_config.json",
@@ -25,7 +26,13 @@ const REQUIRED_MODEL_FILES = [
*/
export type AsrPipeline = (
audio: Float32Array,
- options?: Record<string, unknown>,
+ options?: {
+ return_timestamps?: boolean | "word";
+ chunk_length_s?: number;
+ stride_length_s?: number;
+ language?: string;
+ task?: "transcribe" | "translate";
+ },
) => Promise<{
text: string;
chunks?: Array<{ timestamp: [number, number | null]; text: string }>;
@@ -49,7 +56,7 @@ let pipePromise: Promise<AsrPipeline | null> | null = null;
/** The model id transcripts are tagged with. */
export function narrationModelId(): string {
- return MODEL_ID;
+ return NARRATION_MODEL_ID;
}
export function narrationModelCacheDir(): string {
@@ -58,9 +65,26 @@ export function narrationModelCacheDir(): string {
return path.join(app.getPath("userData"), "models");
}
+export function isNarrationModelCachedAt(
+ cacheDir: string,
+ modelId = NARRATION_MODEL_ID,
+): boolean {
+ const root = path.join(cacheDir, ...modelId.split("/"));
+ return NARRATION_MODEL_FILES.every((file) => existsSync(path.join(root, file)));
+}
+
export function isNarrationModelCached(): boolean {
- const root = path.join(narrationModelCacheDir(), ...MODEL_ID.split("/"));
- return REQUIRED_MODEL_FILES.every((file) => existsSync(path.join(root, file)));
+ return isNarrationModelCachedAt(narrationModelCacheDir());
+}
+
+/** Remove the superseded English-only weights only after the multilingual model is ready. */
+export async function removeLegacyNarrationModelCache(
+ cacheDir = narrationModelCacheDir(),
+): Promise<void> {
+ await rm(path.join(cacheDir, ...LEGACY_NARRATION_MODEL_ID.split("/")), {
+ recursive: true,
+ force: true,
+ });
}
async function build(options: WhisperLoadOptions): Promise<AsrPipeline> {
@@ -73,7 +97,7 @@ async function build(options: WhisperLoadOptions): Promise<AsrPipeline> {
// every run after is offline from this cache. Nothing is bundled with the app.
tf.env.cacheDir = narrationModelCacheDir();
- const pipe = await tf.pipeline("automatic-speech-recognition", MODEL_ID, {
+ const pipe = await tf.pipeline("automatic-speech-recognition", NARRATION_MODEL_ID, {
dtype: DTYPE,
local_files_only: !options.allowDownload,
progress_callback: options.onProgress,
diff --git a/electron/preload.cjs b/electron/preload.cjs
index 6db5f3d..28bbe49 100644
--- a/electron/preload.cjs
+++ b/electron/preload.cjs
@@ -7,6 +7,7 @@ const IPC = {
stop: "recorder:stop",
discard: "recorder:discard",
microphone: "recorder:microphone",
+ narrationLanguage: "recorder:narration-language",
status: "recorder:status",
marker: "recorder:marker",
doctor: "doctor:check",
@@ -45,6 +46,7 @@ contextBridge.exposeInMainWorld("skillRecorder", {
stop: () => ipcRenderer.invoke(IPC.stop),
discard: () => ipcRenderer.invoke(IPC.discard),
setMicrophoneEnabled: (enabled) => ipcRenderer.invoke(IPC.microphone, enabled),
+ setNarrationLanguage: (language) => ipcRenderer.invoke(IPC.narrationLanguage, language),
status: () => ipcRenderer.invoke(IPC.status),
marker: (note) => ipcRenderer.invoke(IPC.marker, note),
doctor: () => ipcRenderer.invoke(IPC.doctor),
diff --git a/electron/recorder/controller.test.ts b/electron/recorder/controller.test.ts
index f5fb32e..e20bb2b 100644
--- a/electron/recorder/controller.test.ts
+++ b/electron/recorder/controller.test.ts
@@ -5,15 +5,22 @@ import path from "node:path";
import test from "node:test";
import { FULL_CAPTURE } from "../../common/config";
+import type { NarrationLanguage } from "../../common/narration";
import { RecorderController, type SessionAudioRecorder } from "./controller";
class FakeAudioRecorder implements SessionAudioRecorder {
readonly calls: string[] = [];
finishVideoStartEpoch: number | null | undefined;
+ narrationLanguage: NarrationLanguage | null = null;
failEnable = false;
- async start(): Promise<void> {
+ async start(
+ _sessionDir: string,
+ _sessionStartedAt: number,
+ narrationLanguage: NarrationLanguage,
+ ): Promise<void> {
this.calls.push("start");
+ this.narrationLanguage = narrationLanguage;
}
async enable(): Promise<void> {
@@ -51,6 +58,7 @@ test("microphone toggles are serialized and finalized on save", async () => {
const started = await controller.start();
assert.equal(started.ok, true);
+ assert.equal(audio.narrationLanguage, "en");
assert.equal(controller.status().microphone.state, "off");
assert.equal((await controller.setMicrophoneEnabled(true)).ok, true);
@@ -82,6 +90,31 @@ test("microphone toggles are serialized and finalized on save", async () => {
});
});
+test("preferred narration language is reused by optionless recording starts", async () => {
+ await withSessionsRoot(async () => {
+ const audio = new FakeAudioRecorder();
+ const controller = new RecorderController({
+ resolveConfig: () => ({ ...FULL_CAPTURE, video: false }),
+ buildCollectors: () => [],
+ createAudioRecorder: () => audio,
+ deleteSession: async () => undefined,
+ });
+
+ assert.deepEqual(await controller.setNarrationLanguage("it"), {
+ ok: true,
+ language: "it",
+ });
+ const started = await controller.start({ narration: true });
+ assert.equal(started.ok, true);
+ assert.equal(audio.narrationLanguage, "it");
+ assert.equal(controller.status().narrationLanguage, "it");
+ const changeWhileRecording = await controller.setNarrationLanguage("fr");
+ assert.equal(changeWhileRecording.ok, false);
+ assert.match(changeWhileRecording.error ?? "", /cannot change while recording/i);
+ assert.equal((await controller.stop()).ok, true);
+ });
+});
+
test("discard removes the active session and skips post-processing", async () => {
await withSessionsRoot(async (root) => {
const audio = new FakeAudioRecorder();
diff --git a/electron/recorder/controller.ts b/electron/recorder/controller.ts
index 6b7d315..ce827d7 100644
--- a/electron/recorder/controller.ts
+++ b/electron/recorder/controller.ts
@@ -8,11 +8,17 @@ import type {
DiscardResult,
MarkerResult,
MicrophoneResult,
+ NarrationLanguageResult,
RecorderStatus,
StartOptions,
StartResult,
StopResult,
} from "../../common/ipc";
+import {
+ DEFAULT_NARRATION_LANGUAGE,
+ isNarrationLanguage,
+ type NarrationLanguage,
+} from "../../common/narration";
import type { RecorderState, SessionMeta } from "../../common/types";
import { createLogger } from "../logger";
import type { Collector } from "./collector";
@@ -38,7 +44,11 @@ export interface SessionVideoRecorder {
/** Microphone sidecar that can create multiple capture intervals per session. */
export interface SessionAudioRecorder {
- start(sessionDir: string, sessionStartedAt: number): Promise<void>;
+ start(
+ sessionDir: string,
+ sessionStartedAt: number,
+ narrationLanguage: NarrationLanguage,
+ ): Promise<void>;
enable(): Promise<void>;
disable(): Promise<unknown>;
finish(videoStartEpoch: number | null): Promise<unknown>;
@@ -82,6 +92,7 @@ export class RecorderController {
private audio: SessionAudioRecorder | null = null;
private transition: RecorderStatus["transition"] = "none";
private microphone: RecorderStatus["microphone"] = { state: "off", error: null };
+ private narrationLanguage = DEFAULT_NARRATION_LANGUAGE;
private lastFinish: RecorderStatus["lastFinish"] = null;
private shuttingDown = false;
private operations: Promise<void> = Promise.resolve();
@@ -122,6 +133,7 @@ export class RecorderController {
state: this.state,
sessionId: this.store?.meta.id ?? null,
startedAt: this.store?.meta.startedAt ?? null,
+ narrationLanguage: this.narrationLanguage,
eventCount: this.store?.eventCount ?? 0,
transition: this.transition,
microphone: { ...this.microphone },
@@ -164,6 +176,23 @@ export class RecorderController {
});
}
+ setNarrationLanguage(language: NarrationLanguage): Promise<NarrationLanguageResult> {
+ return this.enqueue(async () => {
+ if (this.store || this.transition !== "none") {
+ return {
+ ok: false,
+ error: "Narration language cannot change while recording.",
+ };
+ }
+ if (!isNarrationLanguage(language)) {
+ return { ok: false, error: "Unsupported narration language." };
+ }
+ this.narrationLanguage = language;
+ this.emit();
+ return { ok: true, language };
+ });
+ }
+
// Marker capture is intentionally retained even though the "Add marker" HUD
// button was removed in favor of voice narration (see docs/future-features.md).
marker(note: string): MarkerResult {
@@ -186,6 +215,15 @@ export class RecorderController {
private async startInternal(options?: StartOptions): Promise<StartResult> {
if (this.store) return { ok: false, error: "Already recording" };
+ if (
+ options?.narrationLanguage !== undefined &&
+ !isNarrationLanguage(options.narrationLanguage)
+ ) {
+ return { ok: false, error: "Unsupported narration language." };
+ }
+ if (options?.narrationLanguage !== undefined) {
+ this.narrationLanguage = options.narrationLanguage;
+ }
this.transition = "starting";
this.microphone = { state: "off", error: null };
@@ -241,7 +279,7 @@ export class RecorderController {
if (this.deps.createAudioRecorder) {
this.audio = this.deps.createAudioRecorder((event) => this.onAudioCaptureEnded(event));
try {
- await this.audio.start(store.dir, store.meta.startedAt);
+ await this.audio.start(store.dir, store.meta.startedAt, this.narrationLanguage);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
log.warn("microphone initialization failed:", message);
diff --git a/electron/sessions.test.ts b/electron/sessions.test.ts
index e87adc3..3d0e0b2 100644
--- a/electron/sessions.test.ts
+++ b/electron/sessions.test.ts
@@ -42,6 +42,7 @@ test("session size includes every artifact and deletion removes the whole direct
);
assert.equal(summary?.sizeBytes, expectedBytes);
assert.equal(summary?.hasAudio, true);
+ assert.equal(summary?.narrationLanguage, "en");
await deleteSession(id);
await assert.rejects(access(dir), { code: "ENOENT" });
diff --git a/electron/sessions.ts b/electron/sessions.ts
index 007ac21..8261378 100644
--- a/electron/sessions.ts
+++ b/electron/sessions.ts
@@ -2,8 +2,13 @@ import { access, lstat, readdir, readFile, rm, stat } from "node:fs/promises";
import path from "node:path";
import { AnalysisSchema, type Analysis } from "../common/analysis";
+import { readAudioNarrationLanguage } from "../common/audio";
import type { SessionSummary } from "../common/ipc";
-import { NARRATION_FILE, type NarrationTranscript } from "../common/narration";
+import {
+ NARRATION_FILE,
+ type NarrationLanguage,
+ type NarrationTranscript,
+} from "../common/narration";
import type { SessionMeta } from "../common/types";
import { createLogger } from "./logger";
import { isValidSessionId, sessionDir, sessionsRoot } from "./recorder/session-store";
@@ -46,6 +51,15 @@ async function loadNarration(
}
}
+async function loadAudioLanguage(dir: string): Promise<NarrationLanguage | null> {
+ try {
+ const raw = JSON.parse(await readFile(path.join(dir, "audio.json"), "utf8")) as unknown;
+ return readAudioNarrationLanguage(raw);
+ } catch {
+ return null;
+ }
+}
+
function isMissing(error: unknown): boolean {
return (
typeof error === "object" &&
@@ -114,12 +128,20 @@ async function summarize(root: string, name: string): Promise<SessionSummary | n
}
if (!meta?.id) return null;
- const [analysis, processed, hasVideo, hasAudio, narration, hasSkill, hasAutomation, sizeBytes] =
- await Promise.all([
+ const [
+ analysis,
+ processed,
+ hasVideo,
+ narrationLanguage,
+ narration,
+ hasSkill,
+ hasAutomation,
+ sizeBytes,
+ ] = await Promise.all([
loadAnalysis(dir),
exists(path.join(dir, "bundle.json")),
exists(path.join(dir, "video.json")),
- exists(path.join(dir, "audio.json")),
+ loadAudioLanguage(dir),
loadNarration(dir),
exists(path.join(dir, "skill.json")),
exists(path.join(dir, "built-automation.json")),
@@ -130,7 +152,7 @@ async function summarize(root: string, name: string): Promise<SessionSummary | n
);
return null;
}),
- ]);
+ ]);
return {
id: meta.id,
@@ -140,7 +162,8 @@ async function summarize(root: string, name: string): Promise<SessionSummary | n
sizeBytes,
processed,
hasVideo,
- hasAudio,
+ hasAudio: narrationLanguage !== null,
+ narrationLanguage,
hasNarration: narration !== null,
narrationSegmentCount: narration?.segmentCount ?? null,
narrationUpdatedAt: narration?.updatedAt ?? null,
diff --git a/package.json b/package.json
index ab18bd9..477ca2c 100644
--- a/package.json
+++ b/package.json
@@ -14,7 +14,7 @@
"build": "tsc --noEmit && vite build",
"typecheck": "tsc --noEmit",
"typecheck:evals": "tsc --noEmit -p evals/tsconfig.json",
- "test": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs --test common/audio.test.ts electron/recording-controls-bounds.test.ts electron/recorder/controller.test.ts electron/frames/extractor.test.ts electron/narration/audio-analysis.test.ts electron/sessions.test.ts",
+ "test": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs --test common/audio.test.ts electron/recording-controls-bounds.test.ts electron/recorder/controller.test.ts electron/frames/extractor.test.ts electron/narration/audio-analysis.test.ts electron/narration/transcribe.test.ts electron/narration/whisper.test.ts electron/sessions.test.ts",
"eval": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs evals/run.ts",
"eval:builder": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs evals/builder/run.ts",
"eval:skill": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs evals/skillbuilder/run.ts",
diff --git a/src/App.css b/src/App.css
index e8004bf..eea1506 100644
--- a/src/App.css
+++ b/src/App.css
@@ -443,11 +443,18 @@ button:focus-visible,
min-height: 15px;
}
+.narration-row {
+ display: flex;
+ align-items: stretch;
+ gap: 8px;
+}
+
.narrate {
display: flex;
align-items: center;
gap: 12px;
- width: 100%;
+ flex: 1;
+ min-width: 0;
text-align: left;
background: transparent;
border: 1px solid var(--line);
@@ -536,6 +543,47 @@ button:focus-visible,
transform: translateX(14px);
}
+.narration-language {
+ display: flex;
+ flex: none;
+ flex-direction: column;
+ justify-content: center;
+ gap: 2px;
+ width: 108px;
+ padding: 7px 9px;
+ border: 1px solid var(--line);
+ border-radius: 12px;
+ background: var(--surface);
+}
+
+.narration-language > span {
+ font-size: 9px;
+ font-weight: 600;
+ color: var(--muted);
+ letter-spacing: 0.06em;
+ text-transform: uppercase;
+}
+
+.narration-language select {
+ width: 100%;
+ padding: 0;
+ border: 0;
+ outline: 0;
+ background: transparent;
+ color: var(--text);
+ font: inherit;
+ font-size: 11px;
+ font-weight: 600;
+}
+
+.narration-language:focus-within {
+ border-color: var(--brand);
+}
+
+.narration-language:has(select:disabled) {
+ opacity: 0.75;
+}
+
.privacy-note {
display: flex;
align-items: center;
diff --git a/src/Library.tsx b/src/Library.tsx
index 5198974..a31e250 100644
--- a/src/Library.tsx
+++ b/src/Library.tsx
@@ -16,6 +16,11 @@ import type {
} from "../common/skill";
import { ARCHITECTURES, TARGETS } from "../common/skill";
import type { AutomationPlan, BuiltAutomation } from "../common/automation";
+import {
+ DEFAULT_NARRATION_LANGUAGE,
+ NARRATION_MODEL_DOWNLOAD_LABEL,
+ narrationLanguageLabel,
+} from "../common/narration";
import {
AnalysisStepTiles,
AutomationStepTiles,
@@ -247,6 +252,9 @@ function AnalysisWorkspace({
}) {
const sessionId = summary.id;
const voicePending = summary.hasAudio && !summary.hasNarration;
+ const voiceLanguage = narrationLanguageLabel(
+ summary.narrationLanguage ?? DEFAULT_NARRATION_LANGUAGE,
+ );
const voiceEmpty = summary.hasNarration && summary.narrationSegmentCount === 0;
const voiceBusy =
narrationStatus?.activeSessionId === sessionId && narrationStatus.phase !== "idle";
@@ -458,10 +466,13 @@ function AnalysisWorkspace({
<span>
{voiceBusy
? narrationWorkLabel(narrationStatus)
- : "Your audio is saved. Transcription runs locally and can be done later."}
+ : `Your audio is saved. Transcription runs locally in ${voiceLanguage} and keeps that language.`}
</span>
{!voiceBusy && narrationStatus?.model !== "ready" && (
- <span>The first use requires a one-time ~250 MB download.</span>
+ <span>
+ The first use requires a one-time {NARRATION_MODEL_DOWNLOAD_LABEL} multilingual
+ model download.
+ </span>
)}
{!voiceBusy && voiceError && (
<span className="voice-card-error">{voiceError}</span>
diff --git a/src/Recorder.tsx b/src/Recorder.tsx
index eaef7f9..e31d567 100644
--- a/src/Recorder.tsx
+++ b/src/Recorder.tsx
@@ -1,6 +1,15 @@
import { useCallback, useEffect, useState } from "react";
import type { DoctorReport, NarrationStatus, RecorderStatus } from "../common/ipc";
+import {
+ DEFAULT_NARRATION_LANGUAGE,
+ isNarrationLanguage,
+ NARRATION_LANGUAGE_CODES,
+ NARRATION_LANGUAGE_LABELS,
+ NARRATION_MODEL_DOWNLOAD_LABEL,
+ narrationLanguageLabel,
+ type NarrationLanguage,
+} from "../common/narration";
import { formatMs } from "./format";
import { WhatsRecorded } from "./WhatsRecorded";
@@ -14,6 +23,8 @@ export function Recorder() {
const [narrationStatus, setNarrationStatus] = useState<NarrationStatus | null>(null);
const [showPrivacy, setShowPrivacy] = useState(false);
const [narrate, setNarrate] = useState(false);
+ const [narrationLanguage, setSelectedNarrationLanguage] =
+ useState<NarrationLanguage>(DEFAULT_NARRATION_LANGUAGE);
const [elapsed, setElapsed] = useState(0);
const [sessionCount, setSessionCount] = useState(0);
const [pendingCount, setPendingCount] = useState(0);
@@ -24,18 +35,23 @@ export function Recorder() {
setPendingCount(list.filter((s) => !s.analysis).length);
}, []);
+ const applyRecorderStatus = useCallback((next: RecorderStatus) => {
+ setStatus(next);
+ setSelectedNarrationLanguage(next.narrationLanguage);
+ }, []);
+
useEffect(() => {
- void window.skillRecorder.status().then(setStatus);
+ void window.skillRecorder.status().then(applyRecorderStatus);
void window.skillRecorder.doctor().then(setDoctor);
void window.skillRecorder.narrationStatus().then(setNarrationStatus);
void refreshCount();
- const offRecorder = window.skillRecorder.onStatusChanged(setStatus);
+ const offRecorder = window.skillRecorder.onStatusChanged(applyRecorderStatus);
const offNarration = window.skillRecorder.onNarrationStatusChanged(setNarrationStatus);
return () => {
offRecorder();
offNarration();
};
- }, [refreshCount]);
+ }, [applyRecorderStatus, refreshCount]);
// The analyze step happens in the library window, so re-check how many
// recordings still need analysis whenever the recorder regains focus.
@@ -68,10 +84,24 @@ export function Recorder() {
const toggle = useCallback(async () => {
const res = recording
? await window.skillRecorder.stop()
- : await window.skillRecorder.start({ narration: narrate });
+ : await window.skillRecorder.start({ narration: narrate, narrationLanguage });
if (!res.ok) window.alert(res.error ?? "Action failed");
- setStatus(await window.skillRecorder.status());
- }, [recording, narrate]);
+ applyRecorderStatus(await window.skillRecorder.status());
+ }, [applyRecorderStatus, recording, narrate, narrationLanguage]);
+
+ const selectNarrationLanguage = useCallback(
+ async (value: string) => {
+ if (!isNarrationLanguage(value)) {
+ window.alert("Unsupported narration language.");
+ return;
+ }
+ setSelectedNarrationLanguage(value);
+ const result = await window.skillRecorder.setNarrationLanguage(value);
+ if (!result.ok) window.alert(result.error ?? "Could not change the narration language.");
+ applyRecorderStatus(await window.skillRecorder.status());
+ },
+ [applyRecorderStatus],
+ );
const openLibrary = useCallback(() => {
void window.skillRecorder.openLibrary();
@@ -118,43 +148,74 @@ export function Recorder() {
</div>
</div>
- <button
- className={`narrate ${narrate ? "on" : ""}`}
- role="switch"
- aria-checked={narrate}
- onClick={() => setNarrate((v) => !v)}
- disabled={recording || transitioning}
- >
- <span className="narrate-icon" aria-hidden>
- <svg width="18" height="18" viewBox="0 0 20 20" fill="none">
- <rect x="7.5" y="2.5" width="5" height="9" rx="2.5" stroke="currentColor" strokeWidth="1.4" />
- <path
- d="M5 9.2a5 5 0 0 0 10 0"
- stroke="currentColor"
- strokeWidth="1.4"
- strokeLinecap="round"
- />
- <path d="M10 14.2v3" stroke="currentColor" strokeWidth="1.4" strokeLinecap="round" />
- </svg>
- </span>
- <span className="narrate-text">
- <span className="narrate-label">Narrate</span>
- <span className="narrate-sub">
- {recording
- ? narrate
- ? "Listening to your voice"
- : "Voice off for this recording"
- : narrate
- ? narrationStatus?.model === "ready"
- ? "Your voice will be included"
- : "Voice saved · transcription needs ~250 MB later"
- : "Explain out loud (optional)"}
+ <div className="narration-row">
+ <button
+ className={`narrate ${narrate ? "on" : ""}`}
+ role="switch"
+ aria-checked={narrate}
+ onClick={() => setNarrate((v) => !v)}
+ disabled={recording || transitioning}
+ >
+ <span className="narrate-icon" aria-hidden>
+ <svg width="18" height="18" viewBox="0 0 20 20" fill="none">
+ <rect
+ x="7.5"
+ y="2.5"
+ width="5"
+ height="9"
+ rx="2.5"
+ stroke="currentColor"
+ strokeWidth="1.4"
+ />
+ <path
+ d="M5 9.2a5 5 0 0 0 10 0"
+ stroke="currentColor"
+ strokeWidth="1.4"
+ strokeLinecap="round"
+ />
+ <path
+ d="M10 14.2v3"
+ stroke="currentColor"
+ strokeWidth="1.4"
+ strokeLinecap="round"
+ />
+ </svg>
</span>
- </span>
- <span className={`narrate-switch ${narrate ? "on" : ""}`} aria-hidden>
- <span className="narrate-knob" />
- </span>
- </button>
+ <span className="narrate-text">
+ <span className="narrate-label">Narrate</span>
+ <span className="narrate-sub">
+ {recording
+ ? narrate
+ ? `Listening · ${narrationLanguageLabel(narrationLanguage)}`
+ : "Voice off for this recording"
+ : narrate
+ ? narrationStatus?.model === "ready"
+ ? "Voice saved · local transcript"
+ : `Voice saved · ${NARRATION_MODEL_DOWNLOAD_LABEL} later`
+ : "Explain out loud (optional)"}
+ </span>
+ </span>
+ <span className={`narrate-switch ${narrate ? "on" : ""}`} aria-hidden>
+ <span className="narrate-knob" />
+ </span>
+ </button>
+
+ <label className="narration-language">
+ <span>Transcript</span>
+ <select
+ value={narrationLanguage}
+ disabled={recording || transitioning}
+ onChange={(event) => void selectNarrationLanguage(event.currentTarget.value)}
+ title="The transcript stays in this language"
+ >
+ {NARRATION_LANGUAGE_CODES.map((language) => (
+ <option key={language} value={language}>
+ {NARRATION_LANGUAGE_LABELS[language]}
+ </option>
+ ))}
+ </select>
+ </label>
+ </div>
<button className="privacy-note" onClick={() => setShowPrivacy(true)}>
<span className="privacy-note-icon" aria-hidden>
@@ -333,13 +394,17 @@ function VoiceModelRow({
return <Row label="voice transcription" status="warn" note="preparing" />;
}
if (status.model === "ready") {
- return <Row label="voice transcription" status="good" note="offline" />;
+ return <Row label="voice transcription" status="good" note="offline · multilingual" />;
}
return (
<Row
label="voice transcription"
status="warn"
- note={status.model === "error" ? "download failed" : "~250 MB"}
+ note={
+ status.model === "error"
+ ? "download failed"
+ : `${NARRATION_MODEL_DOWNLOAD_LABEL} · multilingual`
+ }
action={{
label: status.model === "error" ? "retry" : "download",
disabled: recording,
diff --git a/src/RecordingControls.tsx b/src/RecordingControls.tsx
index e2c09f1..85900a4 100644
--- a/src/RecordingControls.tsx
+++ b/src/RecordingControls.tsx
@@ -1,6 +1,10 @@
import { useCallback, useEffect, useRef, useState } from "react";
import type { RecorderStatus } from "../common/ipc";
+import {
+ DEFAULT_NARRATION_LANGUAGE,
+ narrationLanguageLabel,
+} from "../common/narration";
import { formatMs } from "./format";
export function RecordingControls() {
@@ -91,6 +95,9 @@ export function RecordingControls() {
const lifecycleBusy = finishPending !== null || transitionBusy || !recording;
const microphoneOn = status?.microphone.state === "on";
const microphoneError = status?.microphone.state === "error";
+ const narrationLanguage = narrationLanguageLabel(
+ status?.narrationLanguage ?? DEFAULT_NARRATION_LANGUAGE,
+ );
const microphoneLabel =
status?.microphone.state === "starting"
? "Starting"
@@ -171,13 +178,13 @@ export function RecordingControls() {
disabled={lifecycleBusy || microphoneBusy}
aria-label={
microphoneOn
- ? "Turn microphone off"
+ ? `Turn microphone off. Narration is transcribed in ${narrationLanguage}.`
: microphoneError
? `Retry microphone. ${status?.microphone.error ?? ""}`
- : "Turn microphone on"
+ : `Turn microphone on for ${narrationLanguage} narration`
}
aria-pressed={microphoneOn}
- title={microphoneOn ? "Microphone on" : "Microphone off"}
+ title={`Microphone ${microphoneOn ? "on" : "off"} · ${narrationLanguage} transcript`}
onClick={() => void toggleMicrophone()}
>
<MicrophoneIcon off={!microphoneOn} />
diff --git a/src/WhatsRecorded.tsx b/src/WhatsRecorded.tsx
index 3f2880f..d4b06a2 100644
--- a/src/WhatsRecorded.tsx
+++ b/src/WhatsRecorded.tsx
@@ -82,7 +82,9 @@ export function WhatsRecorded({ onClose }: { onClose: () => void }) {
</li>
<li>
The recording can be turned into text on this computer using an offline model. The
- first transcription needs a one-time ~250 MB download that you choose when to start.
+ transcript stays in the English, Italian, French, or Spanish language you select.
+ The first transcription needs a one-time ~252 MB download that you choose when to
+ start.
</li>
<li>Leave the microphone off and it is never touched.</li>
</ul>