Skip to content

Desktop app that records your on-screen work session and uses the GitHub Copilot SDK to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Autopilots like Scout, Microsoft Copilot Cowork, or Copilot Studio.

Add microphone device selection

Giorgio Ughini04acfa4ff0e3
diff --git a/README.md b/README.md
index 9c542e9..68b42fa 100644
--- a/README.md
+++ b/README.md
@@ -17,8 +17,8 @@ can repeat.
1. **Record** — hit record (or `⌘⇧R` / `Ctrl+Shift+R` from anywhere) and do your task.
Skill Recorder captures the screen and your activity in the background.
2. **Control** — while recording, a movable always-on-top bar shows capture and
- microphone state. Use it to toggle the microphone, finish, or discard the
- recording; discard always asks for confirmation.
+ microphone state. Use its split microphone control to mute, unmute, or change
+ inputs, then finish or discard the recording; discard always asks for confirmation.
3. **Analyze** — on stop, it correlates the signals and a Copilot agent reconstructs
*what you did*: one overall intent plus an ordered list of steps. You can review
and edit the result.
@@ -42,10 +42,12 @@ activity" panel shows exactly what's collected:
it and retained only when the screen changes or a heartbeat is due.
- **Clipboard** — copy/paste content that ties steps together.
- **Narration** *(optional)* — turn on **Narrate** before capture or toggle the
- microphone from the floating recording bar. Each microphone-on interval is
- timestamped on the video timeline, saved locally, and can be transcribed
- **on-device** (Whisper via transformers.js). The first transcription uses an
- explicit, one-time ~250 MB model download.
+ microphone from the floating recording bar. Narrate shows the active input and
+ remembers a selected microphone, with an explicit **System default** fallback.
+ During capture, the bar can switch inputs without interrupting screen recording;
+ each microphone-on interval is timestamped on the video timeline, saved locally,
+ and can be transcribed **on-device** (Whisper via transformers.js). The first
+ transcription uses an explicit, one-time ~250 MB model download.
## Requirements
@@ -58,8 +60,9 @@ activity" panel shows exactly what's collected:
audio decoding. A system `ffmpeg` is used only when opening a recording created by
an older Skill Recorder version that has no snapshot manifest.
-On first launch macOS will prompt for **Screen Recording** (required) and, if you use
-Narrate, **Microphone** permission.
+On first launch macOS will prompt for **Screen Recording** (required). Enabling
+Narrate requests **Microphone** permission immediately so named inputs can be selected
+before recording; the permission-check stream is released without saving audio.
## Run it (development)
diff --git a/common/ipc.ts b/common/ipc.ts
index fca66e5..368c041 100644
--- a/common/ipc.ts
+++ b/common/ipc.ts
@@ -1,5 +1,6 @@
import type { Analysis, AnalysisFeedback, AnalysisStep, Confidence } from "./analysis";
import type { AutomationPlan, BuiltAutomation } from "./automation";
+import type { MicrophoneDevice } from "./microphone";
import type { BuiltSkill, SkillArchitecture, SkillPlan } from "./skill";
import type { RecorderState } from "./types";
@@ -72,6 +73,7 @@ export interface RecorderStatus {
microphone: {
state: "off" | "starting" | "on" | "stopping" | "error";
error: string | null;
+ activeDevice: MicrophoneDevice | null;
};
lastFinish: {
sessionId: string;
@@ -190,6 +192,8 @@ export interface StartResult {
export interface StartOptions {
/** Capture microphone narration for this session (opt-in, off by default). */
narration?: boolean;
+ /** Device selected for the initial narration segment; defaults to the OS input. */
+ microphoneDeviceId?: string;
}
export interface StopResult {
@@ -211,6 +215,30 @@ export interface MicrophoneResult {
error?: string;
}
+export type MicrophonePermissionState = "unknown" | "granted" | "denied";
+
+/** Shared pre-recording microphone preference and current device catalog. */
+export interface MicrophoneSettingsStatus {
+ narrationEnabled: boolean;
+ permission: MicrophonePermissionState;
+ devices: MicrophoneDevice[];
+ /** The preferred device, retained even while it is disconnected. */
+ preferredDeviceId: string;
+ preferredDeviceLabel: string;
+ /** The device that will actually be requested for the next microphone segment. */
+ selectedDeviceId: string;
+ selectedDeviceLabel: string;
+ preferredDeviceUnavailable: boolean;
+ fallback: string | null;
+ error: string | null;
+}
+
+export interface MicrophoneSettingsResult {
+ ok: boolean;
+ status: MicrophoneSettingsStatus;
+ error?: string;
+}
+
export interface MarkerResult {
ok: boolean;
error?: string;
@@ -267,6 +295,10 @@ export const IPC = {
stop: "recorder:stop",
discard: "recorder:discard",
microphone: "recorder:microphone",
+ microphoneSettings: "microphone:settings",
+ microphoneNarration: "microphone:narration",
+ microphoneDevice: "microphone:device",
+ microphoneSettingsChanged: "microphone:settings-changed",
status: "recorder:status",
marker: "recorder:marker",
doctor: "doctor:check",
@@ -302,10 +334,16 @@ export const IPC = {
/** Shape exposed on `window.skillRecorder` by the preload bridge. */
export interface SkillRecorderApi {
- start(options?: StartOptions): Promise<StartResult>;
+ start(): Promise<StartResult>;
stop(): Promise<StopResult>;
discard(): Promise<DiscardResult>;
setMicrophoneEnabled(enabled: boolean): Promise<MicrophoneResult>;
+ microphoneSettings(): Promise<MicrophoneSettingsStatus>;
+ setNarrationEnabled(enabled: boolean): Promise<MicrophoneSettingsResult>;
+ selectMicrophone(deviceId: string): Promise<MicrophoneSettingsResult>;
+ onMicrophoneSettingsChanged(
+ cb: (status: MicrophoneSettingsStatus) => void,
+ ): () => void;
status(): Promise<RecorderStatus>;
marker(note: string): Promise<MarkerResult>;
doctor(): Promise<DoctorReport>;
@@ -367,6 +405,6 @@ export interface SkillRecorderApi {
openLibrary(): Promise<void>;
/** Close the Sessions library window from within it. */
closeLibrary(): Promise<void>;
- /** Resize the recording-controls window while its confirmation is visible. */
+ /** Resize the recording-controls window while an overlay panel is visible. */
setRecordingControlsExpanded(expanded: boolean): Promise<void>;
}
diff --git a/common/microphone.test.ts b/common/microphone.test.ts
new file mode 100644
index 0000000..07b6a0c
--- /dev/null
+++ b/common/microphone.test.ts
@@ -0,0 +1,43 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import {
+ SYSTEM_DEFAULT_MICROPHONE,
+ resolveMicrophonePreference,
+ type MicrophoneDevice,
+} from "./microphone";
+
+const devices: MicrophoneDevice[] = [
+ SYSTEM_DEFAULT_MICROPHONE,
+ { id: "built-in", label: "MacBook Microphone", groupId: "internal" },
+ { id: "usb-new", label: "Desk Mic", groupId: "usb-group" },
+];
+
+test("microphone preference resolves exact and changed device ids", () => {
+ const exact = resolveMicrophonePreference(
+ { id: "built-in", label: "MacBook Microphone", groupId: "internal" },
+ devices,
+ );
+ assert.equal(exact.device.id, "built-in");
+ assert.equal(exact.unavailable, false);
+ assert.equal(exact.changed, false);
+
+ const replugged = resolveMicrophonePreference(
+ { id: "usb-old", label: "Desk Mic", groupId: "usb-group" },
+ devices,
+ );
+ assert.equal(replugged.device.id, "usb-new");
+ assert.equal(replugged.preference.id, "usb-new");
+ assert.equal(replugged.unavailable, false);
+ assert.equal(replugged.changed, true);
+});
+
+test("missing microphone visibly falls back without forgetting the preference", () => {
+ const preference = { id: "missing", label: "Travel Headset", groupId: "headset" };
+ const resolved = resolveMicrophonePreference(preference, devices);
+
+ assert.deepEqual(resolved.device, SYSTEM_DEFAULT_MICROPHONE);
+ assert.deepEqual(resolved.preference, preference);
+ assert.equal(resolved.unavailable, true);
+ assert.equal(resolved.changed, false);
+});
diff --git a/common/microphone.ts b/common/microphone.ts
new file mode 100644
index 0000000..e7fdc0b
--- /dev/null
+++ b/common/microphone.ts
@@ -0,0 +1,95 @@
+export const SYSTEM_DEFAULT_MICROPHONE_ID = "default";
+
+export interface MicrophoneDevice {
+ id: string;
+ label: string;
+ groupId: string;
+}
+
+export interface MicrophonePreference {
+ id: string;
+ label: string;
+ groupId: string;
+}
+
+export const SYSTEM_DEFAULT_MICROPHONE: MicrophoneDevice = {
+ id: SYSTEM_DEFAULT_MICROPHONE_ID,
+ label: "System default",
+ groupId: "",
+};
+
+export interface ResolvedMicrophonePreference {
+ device: MicrophoneDevice;
+ preference: MicrophonePreference;
+ unavailable: boolean;
+ changed: boolean;
+}
+
+/**
+ * Resolve a persisted preference against the current device catalog. Chromium
+ * device ids can change after a replug, so fall back to group id and then a
+ * unique label before using the system default.
+ */
+export function resolveMicrophonePreference(
+ preference: MicrophonePreference,
+ devices: readonly MicrophoneDevice[],
+): ResolvedMicrophonePreference {
+ if (preference.id === SYSTEM_DEFAULT_MICROPHONE_ID) {
+ return {
+ device: SYSTEM_DEFAULT_MICROPHONE,
+ preference: microphonePreference(SYSTEM_DEFAULT_MICROPHONE),
+ unavailable: false,
+ changed: !samePreference(preference, SYSTEM_DEFAULT_MICROPHONE),
+ };
+ }
+
+ const physicalDevices = devices.filter(
+ (device) => device.id !== SYSTEM_DEFAULT_MICROPHONE_ID,
+ );
+ const exact = physicalDevices.find((device) => device.id === preference.id);
+ const groupMatch =
+ exact ??
+ (preference.groupId
+ ? uniqueMatch(physicalDevices.filter((device) => device.groupId === preference.groupId))
+ : null);
+ const labelMatch =
+ groupMatch ??
+ (preference.label
+ ? uniqueMatch(physicalDevices.filter((device) => device.label === preference.label))
+ : null);
+
+ if (labelMatch) {
+ return {
+ device: labelMatch,
+ preference: microphonePreference(labelMatch),
+ unavailable: false,
+ changed: !samePreference(preference, labelMatch),
+ };
+ }
+
+ return {
+ device: SYSTEM_DEFAULT_MICROPHONE,
+ preference,
+ unavailable: true,
+ changed: false,
+ };
+}
+
+export function microphonePreference(device: MicrophoneDevice): MicrophonePreference {
+ return { id: device.id, label: device.label, groupId: device.groupId };
+}
+
+function uniqueMatch(devices: readonly MicrophoneDevice[]): MicrophoneDevice | null {
+ return devices.length === 1 ? devices[0] : null;
+}
+
+function samePreference(
+ preference: MicrophonePreference,
+ device: MicrophoneDevice,
+): boolean {
+ return (
+ preference.id === device.id &&
+ preference.label === device.label &&
+ preference.groupId === device.groupId
+ );
+}
diff --git a/docs/windows-capture.md b/docs/windows-capture.md
index 0633cb2..490173a 100644
--- a/docs/windows-capture.md
+++ b/docs/windows-capture.md
@@ -59,21 +59,24 @@ Run a real recording on Windows and verify each source lands in the session's
preview and hash.
5. **Video.** Confirm `video.webm`, `video-frames.json`, snapshots under
`video-frames/`, and retained images under `frames/`.
-6. **Voice narration.** Turn on the **Narrate** switch before starting or use the
- microphone control in the floating recording bar. Toggle it off and on around
- two spoken sentences. After Stop, confirm `audio.json` contains a version-2
- segment manifest and the referenced `audio/segment-*.webm` files exist. Their
- session/video offsets should preserve the silent gap while the microphone was
- off. If the model is not installed, use **Download &
+6. **Voice narration.** Turn on **Narrate**, grant microphone access, and select a
+ named input or **System default** before starting. During capture, use the main
+ microphone button to mute/unmute and its adjacent menu to switch inputs. Speak
+ before and after a switch, then confirm `audio.json` contains separate version-2
+ segments whose session/video offsets preserve the boundary. Disconnecting the
+ active device must stop microphone capture with a visible error rather than
+ silently recording from another input; reconnecting it should restore the saved
+ preference when Chromium can identify it. If the model is not installed, use **Download &
transcribe** in Sessions (or the actionable HUD readiness row) to approve the
one-time ~250 MB Whisper download. Confirm `narration.json` then contains your
words with `atMs` offsets. Later runs are offline. On Windows the mic grant is
requested by the OS on first use.
7. **Recording controls.** Confirm the floating bar stays above the active app,
- can be dragged without making its buttons unclickable, and does not appear in
+ its microphone menu and discard confirmation expand above the fixed bar, it can
+ be dragged without making its buttons unclickable, and it does not appear in
captured frames where Windows capture protection is supported. Canceling the
- discard confirmation must continue the same recording; confirming it must
- leave no saved session.
+ discard confirmation must continue the same recording; confirming it must leave
+ no saved session.
8. **Stop.** The recording should show up in the library as `recorded`, and
analysis should produce a coherent intent + ordered steps.
diff --git a/electron/audio/capture-preload.cjs b/electron/audio/capture-preload.cjs
index 23b91ed..2c32d24 100644
--- a/electron/audio/capture-preload.cjs
+++ b/electron/audio/capture-preload.cjs
@@ -18,6 +18,53 @@ let requestedStopEpoch = 0;
// cluster is lost and the webm ends prematurely.
let sendChain = Promise.resolve();
+function microphoneConstraints(deviceId) {
+ return {
+ echoCancellation: true,
+ noiseSuppression: true,
+ channelCount: 1,
+ ...(typeof deviceId === "string" && deviceId && deviceId !== "default"
+ ? { deviceId: { exact: deviceId } }
+ : {}),
+ };
+}
+
+function serializeError(error) {
+ return {
+ message: error instanceof Error ? error.message : String(error),
+ name: error && typeof error.name === "string" ? error.name : "",
+ };
+}
+
+async function enumerateMicrophones() {
+ if (!navigator.mediaDevices || !navigator.mediaDevices.enumerateDevices) return [];
+ const devices = await navigator.mediaDevices.enumerateDevices();
+ return devices
+ .filter((device) => device.kind === "audioinput")
+ .map((device) => ({
+ id: device.deviceId,
+ label: device.label,
+ groupId: device.groupId,
+ }));
+}
+
+async function publishMicrophones() {
+ try {
+ ipcRenderer.send("audio:devices", await enumerateMicrophones());
+ } catch (error) {
+ const detail = serializeError(error);
+ ipcRenderer.send("audio:devices", []);
+ ipcRenderer.send("audio:device-error", detail.message, detail.name);
+ }
+}
+
+if (navigator.mediaDevices) {
+ navigator.mediaDevices.addEventListener("devicechange", () => {
+ void publishMicrophones();
+ });
+ void publishMicrophones();
+}
+
function cleanup() {
try {
if (stream) for (const track of stream.getTracks()) track.stop();
@@ -31,7 +78,7 @@ function cleanup() {
}
ipcRenderer.on("audio:start", async (_event, opts) => {
- const { id, bitsPerSecond } = opts || {};
+ const { id, bitsPerSecond, deviceId } = opts || {};
if (typeof id !== "string" || !id) {
ipcRenderer.send("audio:error", "", "Invalid microphone segment id.");
return;
@@ -47,11 +94,7 @@ ipcRenderer.on("audio:start", async (_event, opts) => {
// A plain microphone request. Echo cancellation / noise suppression keep the
// narration clean for the transcriber without us touching the samples.
const nextStream = await navigator.mediaDevices.getUserMedia({
- audio: {
- echoCancellation: true,
- noiseSuppression: true,
- channelCount: 1,
- },
+ audio: microphoneConstraints(deviceId),
video: false,
});
if (captureId !== id) {
@@ -59,6 +102,7 @@ ipcRenderer.on("audio:start", async (_event, opts) => {
return;
}
stream = nextStream;
+ void publishMicrophones();
for (const track of stream.getAudioTracks()) {
track.addEventListener("ended", () => {
if (!requestedStopEpoch) {
@@ -82,7 +126,13 @@ ipcRenderer.on("audio:start", async (_event, opts) => {
});
};
recorder.onstart = () => {
- ipcRenderer.send("audio:started", id, epochNow());
+ const track = stream && stream.getAudioTracks()[0];
+ const settings = track ? track.getSettings() : {};
+ ipcRenderer.send("audio:started", id, epochNow(), {
+ id: settings.deviceId || deviceId || "default",
+ label: (track && track.label) || "Microphone",
+ groupId: settings.groupId || "",
+ });
};
recorder.onstop = () => {
// Wait for every queued chunk (including the final one) to be sent.
@@ -93,18 +143,50 @@ ipcRenderer.on("audio:start", async (_event, opts) => {
});
};
recorder.onerror = (e) => {
- ipcRenderer.send("audio:error", id, String((e && e.error) || e));
+ const detail = serializeError((e && e.error) || e);
+ ipcRenderer.send("audio:error", id, detail.message, detail.name);
};
// Emit a chunk every second so long sessions stream to disk incrementally.
recorder.start(1000);
} catch (err) {
if (captureId === id) cleanup();
- ipcRenderer.send("audio:error", id, err instanceof Error ? err.message : String(err));
+ const detail = serializeError(err);
+ ipcRenderer.send("audio:error", id, detail.message, detail.name);
ipcRenderer.send("audio:stopped", id, epochNow());
}
});
+ipcRenderer.on("audio:probe", async (_event, opts) => {
+ const { requestId, deviceId } = opts || {};
+ if (typeof requestId !== "string" || !requestId) return;
+ let probeStream = null;
+ try {
+ if (recorder && recorder.state !== "inactive") {
+ throw new Error("The microphone is already recording.");
+ }
+ probeStream = await navigator.mediaDevices.getUserMedia({
+ audio: microphoneConstraints(deviceId),
+ video: false,
+ });
+ for (const track of probeStream.getTracks()) track.stop();
+ probeStream = null;
+ const devices = await enumerateMicrophones();
+ ipcRenderer.send("audio:probe-result", requestId, { ok: true, devices });
+ ipcRenderer.send("audio:devices", devices);
+ } catch (error) {
+ if (probeStream) {
+ for (const track of probeStream.getTracks()) track.stop();
+ }
+ const detail = serializeError(error);
+ ipcRenderer.send("audio:probe-result", requestId, {
+ ok: false,
+ error: detail.message,
+ errorName: detail.name,
+ });
+ }
+});
+
ipcRenderer.on("audio:stop", (_event, opts) => {
const id = opts && opts.id;
if (typeof id !== "string" || id !== captureId) return;
@@ -121,7 +203,8 @@ ipcRenderer.on("audio:stop", (_event, opts) => {
} catch (err) {
const stopEpoch = requestedStopEpoch || epochNow();
cleanup();
- ipcRenderer.send("audio:error", id, err instanceof Error ? err.message : String(err));
+ const detail = serializeError(err);
+ ipcRenderer.send("audio:error", id, detail.message, detail.name);
ipcRenderer.send("audio:stopped", id, stopEpoch);
}
});
diff --git a/electron/audio/recorder.ts b/electron/audio/recorder.ts
index fa19bf9..0dc8383 100644
--- a/electron/audio/recorder.ts
+++ b/electron/audio/recorder.ts
@@ -1,9 +1,10 @@
+import { randomUUID } from "node:crypto";
import { createWriteStream, existsSync, type WriteStream } from "node:fs";
-import { mkdir, rename, rm, unlink, writeFile } from "node:fs/promises";
+import { mkdir, readFile, rename, rm, unlink, writeFile } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
-import { BrowserWindow, ipcMain, type IpcMainEvent } from "electron";
+import { app, BrowserWindow, ipcMain, type IpcMainEvent } from "electron";
import {
AUDIO_MANIFEST_VERSION,
@@ -12,8 +13,25 @@ import {
type AudioSegment,
type LegacyAudioMetadata,
} from "../../common/audio";
+import type {
+ MicrophonePermissionState,
+ MicrophoneSettingsResult,
+ MicrophoneSettingsStatus,
+ StartOptions,
+} from "../../common/ipc";
+import {
+ microphonePreference,
+ resolveMicrophonePreference,
+ SYSTEM_DEFAULT_MICROPHONE,
+ SYSTEM_DEFAULT_MICROPHONE_ID,
+ type MicrophoneDevice,
+ type MicrophonePreference,
+} from "../../common/microphone";
import { createLogger } from "../logger";
-import type { AudioCaptureEnded } from "../recorder/controller";
+import type {
+ AudioCaptureEnded,
+ SessionAudioRecorder,
+} from "../recorder/controller";
const log = createLogger("Narration/audio");
const dirname = path.dirname(fileURLToPath(import.meta.url));
@@ -25,7 +43,10 @@ export type AudioResult = LegacyAudioMetadata;
const BITS_PER_SECOND = 24_000;
const START_TIMEOUT_MS = 10_000;
const STOP_TIMEOUT_MS = 4_000;
+const PROBE_TIMEOUT_MS = 10_000;
const AUDIO_DIR = "audio";
+const PREFERENCES_FILE = "microphone-preferences.json";
+const SERVICE_UNAVAILABLE = "The microphone service is unavailable.";
interface PendingSegment {
id: string;
@@ -39,125 +60,260 @@ interface PendingSegment {
error: string | null;
ended: boolean;
stopRequested: boolean;
- startedResolve: (() => void) | null;
+ requestedDeviceId: string;
+ device: MicrophoneDevice | null;
+ startedResolve: ((device: MicrophoneDevice) => void) | null;
startedReject: ((error: Error) => void) | null;
stoppedResolve: (() => void) | null;
}
+interface StoredPreferences {
+ narrationEnabled: boolean;
+ microphone: MicrophonePreference;
+}
+
+interface ProbeOutcome {
+ ok: boolean;
+ error?: string;
+}
+
+interface PendingProbe {
+ resolve: (outcome: ProbeOutcome) => void;
+ timeout: NodeJS.Timeout;
+}
+
/**
- * Owns microphone capture for one recording session. The hidden renderer stays
- * alive for the session, but each microphone-on interval gets a fresh
- * MediaStream, MediaRecorder, and WebM file so off-periods remain real timeline
- * gaps and the OS microphone device is released between intervals.
+ * App-lifetime microphone service plus one-at-a-time session writer. Keeping the
+ * hidden renderer alive makes device ids consistent between preflight selection
+ * and the MediaRecorder stream that ultimately uses them.
*/
export class AudioRecorder {
private win: BrowserWindow | null = null;
+ private readyTask: Promise<void> | null = null;
+ private disposed = false;
+ private ipcRegistered = false;
+ private preferencesLoaded = false;
+ private preferencesFile = "";
+ private preferenceWrites: Promise<void> = Promise.resolve();
+ private preferences: StoredPreferences = {
+ narrationEnabled: false,
+ microphone: microphonePreference(SYSTEM_DEFAULT_MICROPHONE),
+ };
+ private permission: MicrophonePermissionState = "unknown";
+ private devices: MicrophoneDevice[] = [SYSTEM_DEFAULT_MICROPHONE];
+ private settingsError: string | null = null;
+ private readonly probes = new Map<string, PendingProbe>();
+ private readonly settingsOperations = new Set<Promise<unknown>>();
+
private dir = "";
private sessionStartedAt = 0;
private sequence = 0;
private active: PendingSegment | null = null;
private disableTask: Promise<AudioSegment | null> | null = null;
private readonly segments: AudioSegment[] = [];
+ private onCaptureEnded: (event: AudioCaptureEnded) => void = () => undefined;
constructor(
- private readonly onCaptureEnded: (event: AudioCaptureEnded) => void = () => undefined,
+ private readonly onSettingsChanged: (
+ status: MicrophoneSettingsStatus,
+ ) => void = () => undefined,
) {}
- private readonly onChunk = (event: IpcMainEvent, id: string, chunk: Uint8Array) => {
- const active = this.ownedSegment(event, id);
- if (!active || !(chunk instanceof Uint8Array) || chunk.byteLength === 0) return;
- const buffer = Buffer.from(chunk);
- active.bytes += buffer.byteLength;
- active.stream.write(buffer);
- };
+ createSession(
+ onCaptureEnded: (event: AudioCaptureEnded) => void,
+ ): SessionAudioRecorder {
+ return {
+ start: (sessionDir, sessionStartedAt) =>
+ this.startSession(sessionDir, sessionStartedAt, onCaptureEnded),
+ enable: (deviceId) => this.enable(deviceId),
+ disable: () => this.disable(),
+ finish: (videoStartEpoch) => this.finishSession(videoStartEpoch),
+ };
+ }
- private readonly onStarted = (event: IpcMainEvent, id: string, epoch: number) => {
- const active = this.ownedSegment(event, id);
- if (!active || !Number.isFinite(epoch)) return;
- active.startEpoch = epoch;
- active.startedResolve?.();
- active.startedResolve = null;
- active.startedReject = null;
- log.info("microphone segment started; anchor epoch", epoch);
- };
+ async initialize(): Promise<void> {
+ if (this.disposed) throw new Error("The microphone service has been disposed.");
+ await this.loadPreferences();
+ await this.ensureWindow();
+ this.emitSettings();
+ }
- private readonly onStopped = (event: IpcMainEvent, id: string, epoch: number) => {
- const active = this.ownedSegment(event, id);
- if (!active) return;
- const unexpected = !active.stopRequested && active.startEpoch != null;
- active.stopEpoch = Number.isFinite(epoch) ? epoch : Date.now();
- active.ended = true;
- active.stoppedResolve?.();
- active.stoppedResolve = null;
- if (unexpected) {
- this.finishUnexpected(active, "The microphone stopped unexpectedly.", false);
+ dispose(): void {
+ if (this.disposed) return;
+ this.disposed = true;
+ for (const probe of this.probes.values()) {
+ clearTimeout(probe.timeout);
+ probe.resolve({ ok: false, error: "Skill Recorder is shutting down." });
}
- };
+ this.probes.clear();
+ if (this.ipcRegistered) {
+ ipcMain.removeListener("audio:chunk", this.onChunk);
+ ipcMain.removeListener("audio:started", this.onStarted);
+ ipcMain.removeListener("audio:stopped", this.onStopped);
+ ipcMain.removeListener("audio:error", this.onError);
+ ipcMain.removeListener("audio:devices", this.onDevices);
+ ipcMain.removeListener("audio:device-error", this.onDeviceError);
+ ipcMain.removeListener("audio:probe-result", this.onProbeResult);
+ this.ipcRegistered = false;
+ }
+ if (this.win && !this.win.isDestroyed()) this.win.destroy();
+ this.win = null;
+ this.readyTask = null;
+ }
- private readonly onError = (event: IpcMainEvent, id: string, error: string) => {
- const active = this.ownedSegment(event, id);
- if (!active) return;
- const reason = error || "Microphone capture failed.";
- active.error = reason;
- log.warn("microphone unavailable:", reason);
- if (active.startEpoch == null) {
- active.failed = true;
- active.startedReject?.(new Error(reason));
- active.startedResolve = null;
- active.startedReject = null;
- return;
+ settings(): MicrophoneSettingsStatus {
+ const resolved = resolveMicrophonePreference(
+ this.preferences.microphone,
+ this.devices,
+ );
+ const preferredLabel =
+ this.preferences.microphone.label || "Selected microphone";
+ return {
+ narrationEnabled: this.preferences.narrationEnabled,
+ permission: this.permission,
+ devices: this.devices.map((device) => ({ ...device })),
+ preferredDeviceId: this.preferences.microphone.id,
+ preferredDeviceLabel: preferredLabel,
+ selectedDeviceId: resolved.device.id,
+ selectedDeviceLabel: resolved.device.label,
+ preferredDeviceUnavailable: resolved.unavailable,
+ fallback: resolved.unavailable
+ ? `${preferredLabel} is unavailable. System default will be used.`
+ : null,
+ error: this.settingsError,
+ };
+ }
+
+ startOptions(): StartOptions {
+ return {
+ narration: this.preferences.narrationEnabled,
+ microphoneDeviceId: this.effectiveDeviceId(),
+ };
+ }
+
+ effectiveDeviceId(): string {
+ return resolveMicrophonePreference(
+ this.preferences.microphone,
+ this.devices,
+ ).device.id;
+ }
+
+ async setNarrationEnabled(
+ enabled: boolean,
+ ): Promise<MicrophoneSettingsResult> {
+ return this.trackSettingsOperation(
+ this.setNarrationEnabledInternal(enabled),
+ );
+ }
+
+ selectDevice(deviceId: string): Promise<MicrophoneSettingsResult> {
+ return this.trackSettingsOperation(this.selectDeviceInternal(deviceId));
+ }
+
+ async whenSettingsSettled(): Promise<void> {
+ while (this.settingsOperations.size > 0) {
+ await Promise.allSettled([...this.settingsOperations]);
}
- this.finishUnexpected(active, reason, true);
- };
+ }
- /** Prepare the session's hidden audio utility window without touching the mic. */
- async start(sessionDir: string, sessionStartedAt: number): Promise<void> {
- if (this.win) throw new Error("Microphone recorder is already initialized.");
- this.dir = sessionDir;
- this.sessionStartedAt = sessionStartedAt;
- this.sequence = 0;
- this.segments.length = 0;
- await mkdir(path.join(this.dir, AUDIO_DIR), { recursive: true });
+ private async setNarrationEnabledInternal(
+ enabled: boolean,
+ ): Promise<MicrophoneSettingsResult> {
+ try {
+ await this.loadPreferences();
+ if (enabled) await this.ensureWindow();
+ } catch (error) {
+ return this.settingsFailure(error);
+ }
- ipcMain.on("audio:chunk", this.onChunk);
- ipcMain.on("audio:started", this.onStarted);
- ipcMain.on("audio:stopped", this.onStopped);
- ipcMain.on("audio:error", this.onError);
+ if (enabled && !this.preferences.narrationEnabled) {
+ const probe = await this.requestPermission();
+ if (!probe.ok) {
+ return {
+ ok: false,
+ status: this.settings(),
+ error: probe.error ?? "Microphone permission was not granted.",
+ };
+ }
+ }
- this.win = new BrowserWindow({
- show: false,
- webPreferences: {
- preload: path.join(dirname, "audio", "capture-preload.cjs"),
- contextIsolation: true,
- nodeIntegration: false,
- sandbox: false,
- backgroundThrottling: false,
- },
- });
- this.win.once("closed", () => {
- const active = this.active;
- if (!active || active.ended) return;
- active.failed = true;
- active.error = "Microphone capture window closed unexpectedly.";
- active.startedReject?.(new Error(active.error));
- active.ended = true;
- active.stoppedResolve?.();
- this.finishUnexpected(active, active.error, true);
- });
+ const previous = this.preferences.narrationEnabled;
+ this.preferences.narrationEnabled = enabled;
+ this.settingsError = null;
+ try {
+ await this.persistPreferences();
+ } catch (error) {
+ this.preferences.narrationEnabled = previous;
+ return this.settingsFailure(error, "Could not save the narration preference");
+ }
+ this.emitSettings();
+ return { ok: true, status: this.settings() };
+ }
+ private async selectDeviceInternal(
+ deviceId: string,
+ ): Promise<MicrophoneSettingsResult> {
try {
- await this.win.loadFile(path.join(dirname, "audio", "capture.html"));
+ await this.initialize();
} catch (error) {
- await this.teardown();
- throw error;
+ return this.settingsFailure(error);
+ }
+
+ const device = this.devices.find((candidate) => candidate.id === deviceId);
+ if (!device) {
+ const error = "That microphone is no longer available.";
+ this.settingsError = error;
+ this.emitSettings();
+ return { ok: false, status: this.settings(), error };
+ }
+
+ const previous = this.preferences.microphone;
+ this.preferences.microphone = microphonePreference(device);
+ this.settingsError = null;
+ try {
+ await this.persistPreferences();
+ } catch (error) {
+ this.preferences.microphone = previous;
+ return this.settingsFailure(error, "Could not save the microphone selection");
}
+ this.emitSettings();
+ return { ok: true, status: this.settings() };
+ }
+
+ private trackSettingsOperation<T>(operation: Promise<T>): Promise<T> {
+ this.settingsOperations.add(operation);
+ void operation.then(
+ () => this.settingsOperations.delete(operation),
+ () => this.settingsOperations.delete(operation),
+ );
+ return operation;
+ }
+
+ private async startSession(
+ sessionDir: string,
+ sessionStartedAt: number,
+ onCaptureEnded: (event: AudioCaptureEnded) => void,
+ ): Promise<void> {
+ if (this.dir) throw new Error("Microphone recorder is already initialized.");
+ await this.initialize();
+ await mkdir(path.join(sessionDir, AUDIO_DIR), { recursive: true });
+ this.dir = sessionDir;
+ this.sessionStartedAt = sessionStartedAt;
+ this.sequence = 0;
+ this.segments.length = 0;
+ this.onCaptureEnded = onCaptureEnded;
}
/** Request microphone access and begin a new independently timestamped segment. */
- async enable(): Promise<void> {
- if (!this.win || this.win.isDestroyed()) {
+ private async enable(
+ deviceId = SYSTEM_DEFAULT_MICROPHONE_ID,
+ ): Promise<MicrophoneDevice> {
+ if (!this.dir) {
throw new Error("Microphone capture is not available for this recording.");
}
+ if (!this.win || this.win.isDestroyed()) {
+ throw new Error("The microphone service is unavailable.");
+ }
if (this.active) throw new Error("Microphone capture is already active.");
const number = ++this.sequence;
@@ -177,6 +333,8 @@ export class AudioRecorder {
error: null,
ended: false,
stopRequested: false,
+ requestedDeviceId: deviceId,
+ device: null,
startedResolve: null,
startedReject: null,
stoppedResolve: null,
@@ -186,21 +344,28 @@ export class AudioRecorder {
active.error = error.message;
active.startedReject?.(error);
log.warn("microphone segment write failed:", error.message);
- if (active.startEpoch != null) this.finishUnexpected(active, error.message, true);
+ if (active.startEpoch != null) {
+ this.finishUnexpected(active, error.message, true);
+ }
});
this.active = active;
- const started = new Promise<void>((resolve, reject) => {
+ const started = new Promise<MicrophoneDevice>((resolve, reject) => {
active.startedResolve = resolve;
active.startedReject = reject;
});
this.win.webContents.send("audio:start", {
id,
bitsPerSecond: BITS_PER_SECOND,
+ deviceId,
});
try {
- await withTimeout(started, START_TIMEOUT_MS, "Timed out starting microphone capture.");
+ return await withTimeout(
+ started,
+ START_TIMEOUT_MS,
+ "Timed out starting microphone capture.",
+ );
} catch (error) {
await this.disable();
throw error;
@@ -208,7 +373,7 @@ export class AudioRecorder {
}
/** Flush and close the active segment, releasing the microphone device. */
- async disable(): Promise<AudioSegment | null> {
+ private async disable(): Promise<AudioSegment | null> {
if (this.disableTask) return this.disableTask;
const task = this.disableInternal();
this.disableTask = task;
@@ -238,11 +403,14 @@ export class AudioRecorder {
}
try {
- await withTimeout(stopped, STOP_TIMEOUT_MS, "Timed out stopping microphone capture.");
+ await withTimeout(
+ stopped,
+ STOP_TIMEOUT_MS,
+ "Timed out stopping microphone capture.",
+ );
} catch (error) {
log.warn(error instanceof Error ? error.message : String(error));
active.stopEpoch ??= Date.now();
- if (this.win && !this.win.isDestroyed()) this.win.destroy();
}
}
@@ -256,7 +424,9 @@ export class AudioRecorder {
active.stopEpoch == null
) {
if (existsSync(active.file)) await unlink(active.file).catch(() => undefined);
- if (active.error) log.warn("discarded unusable microphone segment:", active.error);
+ if (active.error) {
+ log.warn("discarded unusable microphone segment:", active.error);
+ }
return null;
}
@@ -282,12 +452,16 @@ export class AudioRecorder {
}
/** Finish the session and atomically persist its segment manifest. */
- async finish(videoStartEpoch: number | null): Promise<AudioManifestV2 | null> {
+ private async finishSession(
+ videoStartEpoch: number | null,
+ ): Promise<AudioManifestV2 | null> {
if (this.active) await this.disable();
const segments = alignAudioSegmentsToVideo(this.segments, videoStartEpoch);
const manifest: AudioManifestV2 | null =
- segments.length > 0 ? { version: AUDIO_MANIFEST_VERSION, segments } : null;
+ segments.length > 0
+ ? { version: AUDIO_MANIFEST_VERSION, segments }
+ : null;
try {
if (manifest) {
@@ -299,21 +473,283 @@ export class AudioRecorder {
} finally {
await rm(temp, { force: true });
}
- } else {
+ } else if (this.dir) {
await rm(path.join(this.dir, "audio.json"), { force: true });
- await rm(path.join(this.dir, AUDIO_DIR), { recursive: true, force: true });
+ await rm(path.join(this.dir, AUDIO_DIR), {
+ recursive: true,
+ force: true,
+ });
}
return manifest;
} finally {
- await this.teardown();
+ this.resetSession();
}
}
+ private readonly onChunk = (
+ event: IpcMainEvent,
+ id: string,
+ chunk: Uint8Array,
+ ) => {
+ const active = this.ownedSegment(event, id);
+ if (!active || !(chunk instanceof Uint8Array) || chunk.byteLength === 0) return;
+ const buffer = Buffer.from(chunk);
+ active.bytes += buffer.byteLength;
+ active.stream.write(buffer);
+ };
+
+ private readonly onStarted = (
+ event: IpcMainEvent,
+ id: string,
+ epoch: number,
+ rawDevice: unknown,
+ ) => {
+ const active = this.ownedSegment(event, id);
+ if (!active || !Number.isFinite(epoch)) return;
+ const device = readActiveDevice(rawDevice, active.requestedDeviceId);
+ active.startEpoch = epoch;
+ active.device = device;
+ active.startedResolve?.(device);
+ active.startedResolve = null;
+ active.startedReject = null;
+ this.permission = "granted";
+ this.settingsError = null;
+ this.emitSettings();
+ log.info("microphone segment started:", device.label, "at", epoch);
+ };
+
+ private readonly onStopped = (
+ event: IpcMainEvent,
+ id: string,
+ epoch: number,
+ ) => {
+ const active = this.ownedSegment(event, id);
+ if (!active) return;
+ const unexpected = !active.stopRequested && active.startEpoch != null;
+ active.stopEpoch = Number.isFinite(epoch) ? epoch : Date.now();
+ active.ended = true;
+ active.stoppedResolve?.();
+ active.stoppedResolve = null;
+ if (unexpected) {
+ this.finishUnexpected(
+ active,
+ "The microphone stopped unexpectedly.",
+ false,
+ );
+ }
+ };
+
+ private readonly onError = (
+ event: IpcMainEvent,
+ id: string,
+ error: string,
+ errorName?: string,
+ ) => {
+ const active = this.ownedSegment(event, id);
+ if (!active) return;
+ const reason = error || "Microphone capture failed.";
+ active.error = reason;
+ if (isPermissionDenied(errorName)) {
+ this.permission = "denied";
+ this.settingsError = reason;
+ this.emitSettings();
+ }
+ log.warn("microphone unavailable:", reason);
+ if (active.startEpoch == null) {
+ active.failed = true;
+ active.startedReject?.(new Error(reason));
+ active.startedResolve = null;
+ active.startedReject = null;
+ return;
+ }
+ this.finishUnexpected(active, reason, true);
+ };
+
+ private readonly onDevices = (event: IpcMainEvent, rawDevices: unknown) => {
+ if (!this.ownsWindow(event)) return;
+ this.applyDevices(rawDevices);
+ };
+
+ private readonly onDeviceError = (
+ event: IpcMainEvent,
+ error: string,
+ errorName?: string,
+ ) => {
+ if (!this.ownsWindow(event)) return;
+ if (isPermissionDenied(errorName)) this.permission = "denied";
+ this.settingsError = error || "Could not list microphones.";
+ this.emitSettings();
+ };
+
+ private readonly onProbeResult = (
+ event: IpcMainEvent,
+ requestId: string,
+ rawResult: unknown,
+ ) => {
+ if (!this.ownsWindow(event) || typeof requestId !== "string") return;
+ const pending = this.probes.get(requestId);
+ if (!pending) return;
+ this.probes.delete(requestId);
+ clearTimeout(pending.timeout);
+
+ const result = readProbeResult(rawResult);
+ if (result.devices) this.applyDevices(result.devices, false);
+ if (result.ok) {
+ this.permission = "granted";
+ this.settingsError = null;
+ this.emitSettings();
+ pending.resolve({ ok: true });
+ return;
+ }
+
+ if (isPermissionDenied(result.errorName)) this.permission = "denied";
+ const error = result.error || "Microphone permission was not granted.";
+ this.settingsError = error;
+ this.emitSettings();
+ pending.resolve({ ok: false, error });
+ };
+
+ private async requestPermission(): Promise<ProbeOutcome> {
+ if (!this.win || this.win.isDestroyed()) {
+ return { ok: false, error: SERVICE_UNAVAILABLE };
+ }
+ const requestId = randomUUID();
+ const outcome = new Promise<ProbeOutcome>((resolve) => {
+ const timeout = setTimeout(() => {
+ this.probes.delete(requestId);
+ const error = "Timed out requesting microphone permission.";
+ this.settingsError = error;
+ this.emitSettings();
+ resolve({ ok: false, error });
+ }, PROBE_TIMEOUT_MS);
+ this.probes.set(requestId, { resolve, timeout });
+ });
+ this.win.webContents.send("audio:probe", {
+ requestId,
+ deviceId: this.effectiveDeviceId(),
+ });
+ return outcome;
+ }
+
+ private applyDevices(rawDevices: unknown, emit = true): void {
+ const normalized = normalizeDevices(rawDevices);
+ this.devices = normalized.devices.map((device) => {
+ if (
+ !normalized.labeledDeviceIds.has(device.id) &&
+ device.id === this.preferences.microphone.id &&
+ this.preferences.microphone.label
+ ) {
+ return {
+ ...device,
+ label: this.preferences.microphone.label,
+ groupId: device.groupId || this.preferences.microphone.groupId,
+ };
+ }
+ return device;
+ });
+ if (normalized.hasLabels) this.permission = "granted";
+ if (this.settingsError === SERVICE_UNAVAILABLE) this.settingsError = null;
+
+ const resolved = resolveMicrophonePreference(
+ this.preferences.microphone,
+ this.devices,
+ );
+ if (resolved.changed) {
+ this.preferences.microphone =
+ normalized.labeledDeviceIds.has(resolved.preference.id) ||
+ resolved.preference.id === this.preferences.microphone.id
+ ? resolved.preference
+ : {
+ ...resolved.preference,
+ label:
+ this.preferences.microphone.label ||
+ resolved.preference.label,
+ groupId:
+ resolved.preference.groupId ||
+ this.preferences.microphone.groupId,
+ };
+ void this.persistPreferences().catch((error) => {
+ this.settingsError = `Could not save the restored microphone selection: ${errorMessage(error)}`;
+ this.emitSettings();
+ });
+ }
+ if (emit) this.emitSettings();
+ }
+
+ private async ensureWindow(): Promise<void> {
+ if (this.win && !this.win.isDestroyed() && this.readyTask) {
+ return this.readyTask;
+ }
+ if (this.disposed) throw new Error("The microphone service has been disposed.");
+ this.registerIpc();
+
+ const win = new BrowserWindow({
+ show: false,
+ webPreferences: {
+ preload: path.join(dirname, "audio", "capture-preload.cjs"),
+ contextIsolation: true,
+ nodeIntegration: false,
+ sandbox: false,
+ backgroundThrottling: false,
+ },
+ });
+ this.win = win;
+ win.once("closed", () => this.handleWindowClosed(win));
+ const ready = win.loadFile(path.join(dirname, "audio", "capture.html"));
+ this.readyTask = ready;
+ try {
+ await ready;
+ } catch (error) {
+ if (!win.isDestroyed()) win.destroy();
+ throw error;
+ }
+ }
+
+ private registerIpc(): void {
+ if (this.ipcRegistered) return;
+ ipcMain.on("audio:chunk", this.onChunk);
+ ipcMain.on("audio:started", this.onStarted);
+ ipcMain.on("audio:stopped", this.onStopped);
+ ipcMain.on("audio:error", this.onError);
+ ipcMain.on("audio:devices", this.onDevices);
+ ipcMain.on("audio:device-error", this.onDeviceError);
+ ipcMain.on("audio:probe-result", this.onProbeResult);
+ this.ipcRegistered = true;
+ }
+
+ private handleWindowClosed(win: BrowserWindow): void {
+ if (this.win !== win) return;
+ this.win = null;
+ this.readyTask = null;
+ if (this.disposed) return;
+
+ this.settingsError = SERVICE_UNAVAILABLE;
+ for (const probe of this.probes.values()) {
+ clearTimeout(probe.timeout);
+ probe.resolve({ ok: false, error: SERVICE_UNAVAILABLE });
+ }
+ this.probes.clear();
+ this.emitSettings();
+
+ const active = this.active;
+ if (!active || active.ended) return;
+ active.failed = true;
+ active.error = "Microphone capture window closed unexpectedly.";
+ active.startedReject?.(new Error(active.error));
+ active.ended = true;
+ active.stoppedResolve?.();
+ this.finishUnexpected(active, active.error, true);
+ }
+
private ownedSegment(event: IpcMainEvent, id: string): PendingSegment | null {
- if (event.sender !== this.win?.webContents || this.active?.id !== id) return null;
+ if (!this.ownsWindow(event) || this.active?.id !== id) return null;
return this.active;
}
+ private ownsWindow(event: IpcMainEvent): boolean {
+ return event.sender === this.win?.webContents;
+ }
+
private finishUnexpected(
active: PendingSegment,
error: string,
@@ -332,18 +768,204 @@ export class AudioRecorder {
.finally(() => this.onCaptureEnded({ error }));
}
- private async teardown(): Promise<void> {
- ipcMain.removeListener("audio:chunk", this.onChunk);
- ipcMain.removeListener("audio:started", this.onStarted);
- ipcMain.removeListener("audio:stopped", this.onStopped);
- ipcMain.removeListener("audio:error", this.onError);
- if (this.active) {
- await closeStream(this.active.stream);
- this.active = null;
+ private resetSession(): void {
+ this.dir = "";
+ this.sessionStartedAt = 0;
+ this.sequence = 0;
+ this.segments.length = 0;
+ this.onCaptureEnded = () => undefined;
+ }
+
+ private async loadPreferences(): Promise<void> {
+ if (this.preferencesLoaded) return;
+ this.preferencesLoaded = true;
+ this.preferencesFile = path.join(app.getPath("userData"), PREFERENCES_FILE);
+ try {
+ const raw = JSON.parse(await readFile(this.preferencesFile, "utf8")) as unknown;
+ this.preferences = readPreferences(raw);
+ } catch (error) {
+ if (isNodeError(error) && error.code === "ENOENT") return;
+ this.settingsError = `Saved microphone preferences could not be read: ${errorMessage(error)}`;
+ log.warn(this.settingsError);
}
- if (this.win && !this.win.isDestroyed()) this.win.destroy();
- this.win = null;
}
+
+ private persistPreferences(): Promise<void> {
+ if (!this.preferencesFile) {
+ this.preferencesFile = path.join(app.getPath("userData"), PREFERENCES_FILE);
+ }
+ const file = this.preferencesFile;
+ const contents = JSON.stringify(this.preferences, null, 2);
+ const write = this.preferenceWrites.then(async () => {
+ await mkdir(path.dirname(file), { recursive: true });
+ const temp = `${file}.tmp.${process.pid}.${Date.now()}`;
+ try {
+ await writeFile(temp, contents);
+ await rename(temp, file);
+ } finally {
+ await rm(temp, { force: true });
+ }
+ });
+ this.preferenceWrites = write.catch(() => undefined);
+ return write;
+ }
+
+ private settingsFailure(
+ error: unknown,
+ prefix = "Microphone setup failed",
+ ): MicrophoneSettingsResult {
+ const message = `${prefix}: ${errorMessage(error)}`;
+ this.settingsError = message;
+ this.emitSettings();
+ return { ok: false, status: this.settings(), error: message };
+ }
+
+ private emitSettings(): void {
+ this.onSettingsChanged(this.settings());
+ }
+}
+
+function normalizeDevices(rawDevices: unknown): {
+ devices: MicrophoneDevice[];
+ hasLabels: boolean;
+ labeledDeviceIds: Set<string>;
+} {
+ if (!Array.isArray(rawDevices)) {
+ return {
+ devices: [SYSTEM_DEFAULT_MICROPHONE],
+ hasLabels: false,
+ labeledDeviceIds: new Set(),
+ };
+ }
+
+ const devices: MicrophoneDevice[] = [SYSTEM_DEFAULT_MICROPHONE];
+ const seen = new Set([SYSTEM_DEFAULT_MICROPHONE_ID]);
+ const labeledDeviceIds = new Set<string>();
+ let unnamed = 0;
+ let hasLabels = false;
+ for (const value of rawDevices) {
+ if (!value || typeof value !== "object") continue;
+ const id = "id" in value && typeof value.id === "string" ? value.id : "";
+ if (!id || seen.has(id)) continue;
+ seen.add(id);
+ const rawLabel =
+ "label" in value && typeof value.label === "string"
+ ? value.label.trim()
+ : "";
+ if (rawLabel) {
+ hasLabels = true;
+ labeledDeviceIds.add(id);
+ }
+ const groupId =
+ "groupId" in value && typeof value.groupId === "string"
+ ? value.groupId
+ : "";
+ devices.push({
+ id,
+ label: rawLabel || `Microphone ${++unnamed}`,
+ groupId,
+ });
+ }
+ return { devices, hasLabels, labeledDeviceIds };
+}
+
+function readActiveDevice(
+ value: unknown,
+ requestedDeviceId: string,
+): MicrophoneDevice {
+ if (value && typeof value === "object") {
+ const id = "id" in value && typeof value.id === "string" ? value.id : "";
+ const label =
+ "label" in value && typeof value.label === "string"
+ ? value.label.trim()
+ : "";
+ const groupId =
+ "groupId" in value && typeof value.groupId === "string"
+ ? value.groupId
+ : "";
+ if (id || label) {
+ return {
+ id: id || requestedDeviceId,
+ label: label || "Microphone",
+ groupId,
+ };
+ }
+ }
+ return requestedDeviceId === SYSTEM_DEFAULT_MICROPHONE_ID
+ ? SYSTEM_DEFAULT_MICROPHONE
+ : { id: requestedDeviceId, label: "Microphone", groupId: "" };
+}
+
+function readProbeResult(value: unknown): {
+ ok: boolean;
+ error?: string;
+ errorName?: string;
+ devices?: unknown;
+} {
+ if (!value || typeof value !== "object") return { ok: false };
+ return {
+ ok: "ok" in value && value.ok === true,
+ error:
+ "error" in value && typeof value.error === "string"
+ ? value.error
+ : undefined,
+ errorName:
+ "errorName" in value && typeof value.errorName === "string"
+ ? value.errorName
+ : undefined,
+ devices: "devices" in value ? value.devices : undefined,
+ };
+}
+
+function readPreferences(value: unknown): StoredPreferences {
+ if (!value || typeof value !== "object") {
+ throw new Error("Expected an object.");
+ }
+ const narrationEnabled =
+ "narrationEnabled" in value && typeof value.narrationEnabled === "boolean"
+ ? value.narrationEnabled
+ : false;
+ const rawMicrophone =
+ "microphone" in value && value.microphone && typeof value.microphone === "object"
+ ? value.microphone
+ : null;
+ if (!rawMicrophone) {
+ return {
+ narrationEnabled,
+ microphone: microphonePreference(SYSTEM_DEFAULT_MICROPHONE),
+ };
+ }
+ const id =
+ "id" in rawMicrophone && typeof rawMicrophone.id === "string"
+ ? rawMicrophone.id
+ : "";
+ if (!id) throw new Error("The saved microphone id is invalid.");
+ return {
+ narrationEnabled,
+ microphone: {
+ id,
+ label:
+ "label" in rawMicrophone && typeof rawMicrophone.label === "string"
+ ? rawMicrophone.label
+ : "",
+ groupId:
+ "groupId" in rawMicrophone && typeof rawMicrophone.groupId === "string"
+ ? rawMicrophone.groupId
+ : "",
+ },
+ };
+}
+
+function isPermissionDenied(errorName: string | undefined): boolean {
+ return errorName === "NotAllowedError" || errorName === "SecurityError";
+}
+
+function isNodeError(error: unknown): error is NodeJS.ErrnoException {
+ return error instanceof Error;
+}
+
+function errorMessage(error: unknown): string {
+ return error instanceof Error ? error.message : String(error);
}
function closeStream(stream: WriteStream): Promise<void> {
@@ -363,7 +985,11 @@ function closeStream(stream: WriteStream): Promise<void> {
});
}
-function withTimeout<T>(promise: Promise<T>, timeoutMs: number, message: string): Promise<T> {
+function withTimeout<T>(
+ promise: Promise<T>,
+ timeoutMs: number,
+ message: string,
+): Promise<T> {
let timeout: NodeJS.Timeout | undefined;
const expired = new Promise<never>((_resolve, reject) => {
timeout = setTimeout(() => reject(new Error(message)), timeoutMs);
diff --git a/electron/ipc.ts b/electron/ipc.ts
index fb629ee..4b8e991 100644
--- a/electron/ipc.ts
+++ b/electron/ipc.ts
@@ -9,10 +9,10 @@ import type {
AutomationCreateResult,
AutomationPlanResult,
DeleteSessionResult,
+ MicrophoneSettingsResult,
SkillBuildInput,
SkillCreateResult,
SkillPlanResult,
- StartOptions,
} from "../common/ipc";
import { IPC } from "../common/ipc";
import type { AutomationPlan } from "../common/automation";
@@ -21,6 +21,7 @@ import { AutomationBuilder, loadPersistedAutomation } from "./automationbuilder/
import { Describer, loadPersistedAnalysis } from "./describer/describer";
import { runDoctor } from "./doctor";
import { createLogger } from "./logger";
+import type { AudioRecorder } from "./audio/recorder";
import type { NarrationManager } from "./narration/manager";
import type { RecorderController } from "./recorder/controller";
import { isValidSessionId } from "./recorder/session-store";
@@ -36,12 +37,69 @@ export function registerIpc(
builder: SkillBuilder,
automationBuilder: AutomationBuilder,
narration: NarrationManager,
+ microphones: AudioRecorder,
): void {
- ipcMain.handle(IPC.start, (_event, options?: StartOptions) => recorder.start(options));
+ ipcMain.handle(IPC.start, async () => {
+ await microphones.whenSettingsSettled();
+ return recorder.start(microphones.startOptions());
+ });
ipcMain.handle(IPC.stop, () => recorder.stop());
ipcMain.handle(IPC.discard, () => recorder.discard());
ipcMain.handle(IPC.microphone, (_event, enabled: boolean) =>
- recorder.setMicrophoneEnabled(enabled),
+ recorder.setMicrophoneEnabled(enabled, microphones.effectiveDeviceId()),
+ );
+ ipcMain.handle(IPC.microphoneSettings, () => microphones.settings());
+ ipcMain.handle(
+ IPC.microphoneNarration,
+ async (_event, enabled: boolean): Promise<MicrophoneSettingsResult> => {
+ if (typeof enabled !== "boolean") {
+ return {
+ ok: false,
+ status: microphones.settings(),
+ error: "Invalid narration preference.",
+ };
+ }
+ if (recorder.state === "recording") {
+ return {
+ ok: false,
+ status: microphones.settings(),
+ error: "Choose the next recording's narration state after this recording ends.",
+ };
+ }
+ return microphones.setNarrationEnabled(enabled);
+ },
+ );
+ ipcMain.handle(
+ IPC.microphoneDevice,
+ async (_event, deviceId: string): Promise<MicrophoneSettingsResult> => {
+ if (typeof deviceId !== "string" || !deviceId) {
+ return {
+ ok: false,
+ status: microphones.settings(),
+ error: "Invalid microphone selection.",
+ };
+ }
+ const previousDeviceId = microphones.effectiveDeviceId();
+ const selected = await microphones.selectDevice(deviceId);
+ if (
+ !selected.ok ||
+ recorder.status().microphone.state !== "on" ||
+ microphones.effectiveDeviceId() === previousDeviceId
+ ) {
+ return selected;
+ }
+ const switched = await recorder.setMicrophoneDevice(
+ microphones.effectiveDeviceId(),
+ );
+ if (!switched.ok) {
+ return {
+ ok: false,
+ status: microphones.settings(),
+ error: switched.error ?? "Could not switch microphones.",
+ };
+ }
+ return { ok: true, status: microphones.settings() };
+ },
);
ipcMain.handle(IPC.status, () => recorder.status());
ipcMain.handle(IPC.marker, (_event, note: string) => recorder.marker(note));
diff --git a/electron/main.ts b/electron/main.ts
index 8fb0f73..790179e 100644
--- a/electron/main.ts
+++ b/electron/main.ts
@@ -36,11 +36,15 @@ let quitTask: Promise<void> | null = null;
const narration = new NarrationManager((status) =>
broadcast(IPC.narrationStatusChanged, status),
);
+const microphones = new AudioRecorder((status) =>
+ broadcast(IPC.microphoneSettingsChanged, status),
+);
const recorder = new RecorderController({
resolveConfig: () => ({ ...FULL_CAPTURE }),
buildCollectors: createCollectors,
createVideoRecorder: () => new VideoRecorder(),
- createAudioRecorder: (onCaptureEnded) => new AudioRecorder(onCaptureEnded),
+ createAudioRecorder: (onCaptureEnded) =>
+ microphones.createSession(onCaptureEnded),
deleteSession,
postProcess: async (dir) => {
await processSession(dir);
@@ -52,6 +56,11 @@ const recorder = new RecorderController({
},
});
+async function startRecording(): Promise<unknown> {
+ await microphones.whenSettingsSettled();
+ return recorder.start(microphones.startOptions());
+}
+
/** Send an event to every live window (recorder HUD + library, if open). */
function broadcast(channel: string, payload: unknown): void {
for (const win of BrowserWindow.getAllWindows()) {
@@ -163,11 +172,26 @@ function clampControlsToDisplay(): void {
}
}
-app.whenReady().then(() => {
+app.whenReady().then(async () => {
if (process.platform === "win32") Menu.setApplicationMenu(null);
narration.initialize();
- registerIpc(recorder, describer, builder, automationBuilder, narration);
+ try {
+ await microphones.initialize();
+ } catch (error) {
+ log.warn(
+ "Microphone service initialization failed:",
+ error instanceof Error ? error.message : error,
+ );
+ }
+ registerIpc(
+ recorder,
+ describer,
+ builder,
+ automationBuilder,
+ narration,
+ microphones,
+ );
log.info("Capture: recording all sources");
ipcMain.handle(IPC.openLibrary, () => openLibrary());
@@ -200,7 +224,12 @@ app.whenReady().then(() => {
screen.on("display-metrics-changed", clampControlsToDisplay);
try {
- createTray(recorder, showRecorderWindow, showRecordingControls);
+ createTray(
+ recorder,
+ startRecording,
+ showRecorderWindow,
+ showRecordingControls,
+ );
} catch (err) {
log.warn("Tray unavailable:", err);
}
@@ -208,7 +237,7 @@ app.whenReady().then(() => {
const toggle = () => {
const status = recorder.status();
if (status.transition !== "none") return;
- void (status.state === "recording" ? recorder.stop() : recorder.start());
+ void (status.state === "recording" ? recorder.stop() : startRecording());
};
if (!globalShortcut.register("CommandOrControl+Shift+R", toggle)) {
log.warn("Global shortcut registration failed");
@@ -252,4 +281,5 @@ app.on("will-quit", () => {
void describer.dispose();
void builder.dispose();
void automationBuilder.dispose();
+ microphones.dispose();
});
diff --git a/electron/preload.cjs b/electron/preload.cjs
index 6db5f3d..634e8f1 100644
--- a/electron/preload.cjs
+++ b/electron/preload.cjs
@@ -7,6 +7,10 @@ const IPC = {
stop: "recorder:stop",
discard: "recorder:discard",
microphone: "recorder:microphone",
+ microphoneSettings: "microphone:settings",
+ microphoneNarration: "microphone:narration",
+ microphoneDevice: "microphone:device",
+ microphoneSettingsChanged: "microphone:settings-changed",
status: "recorder:status",
marker: "recorder:marker",
doctor: "doctor:check",
@@ -41,10 +45,18 @@ const IPC = {
};
contextBridge.exposeInMainWorld("skillRecorder", {
- start: (options) => ipcRenderer.invoke(IPC.start, options),
+ start: () => ipcRenderer.invoke(IPC.start),
stop: () => ipcRenderer.invoke(IPC.stop),
discard: () => ipcRenderer.invoke(IPC.discard),
setMicrophoneEnabled: (enabled) => ipcRenderer.invoke(IPC.microphone, enabled),
+ microphoneSettings: () => ipcRenderer.invoke(IPC.microphoneSettings),
+ setNarrationEnabled: (enabled) => ipcRenderer.invoke(IPC.microphoneNarration, enabled),
+ selectMicrophone: (deviceId) => ipcRenderer.invoke(IPC.microphoneDevice, deviceId),
+ onMicrophoneSettingsChanged: (cb) => {
+ const listener = (_event, status) => cb(status);
+ ipcRenderer.on(IPC.microphoneSettingsChanged, listener);
+ return () => ipcRenderer.removeListener(IPC.microphoneSettingsChanged, listener);
+ },
status: () => ipcRenderer.invoke(IPC.status),
marker: (note) => ipcRenderer.invoke(IPC.marker, note),
doctor: () => ipcRenderer.invoke(IPC.doctor),
diff --git a/electron/recorder/controller.test.ts b/electron/recorder/controller.test.ts
index f5fb32e..e942479 100644
--- a/electron/recorder/controller.test.ts
+++ b/electron/recorder/controller.test.ts
@@ -5,10 +5,12 @@ import path from "node:path";
import test from "node:test";
import { FULL_CAPTURE } from "../../common/config";
+import type { MicrophoneDevice } from "../../common/microphone";
import { RecorderController, type SessionAudioRecorder } from "./controller";
class FakeAudioRecorder implements SessionAudioRecorder {
readonly calls: string[] = [];
+ readonly deviceIds: string[] = [];
finishVideoStartEpoch: number | null | undefined;
failEnable = false;
@@ -16,9 +18,15 @@ class FakeAudioRecorder implements SessionAudioRecorder {
this.calls.push("start");
}
- async enable(): Promise<void> {
+ async enable(deviceId = "default"): Promise<MicrophoneDevice> {
this.calls.push("enable");
+ this.deviceIds.push(deviceId);
if (this.failEnable) throw new Error("Microphone permission denied.");
+ return {
+ id: deviceId,
+ label: deviceId === "default" ? "System microphone" : `Microphone ${deviceId}`,
+ groupId: "",
+ };
}
async disable(): Promise<void> {
@@ -96,11 +104,16 @@ test("discard removes the active session and skips post-processing", async () =>
},
});
- const started = await controller.start({ narration: true });
+ const started = await controller.start({
+ narration: true,
+ microphoneDeviceId: "usb-desk",
+ });
assert.equal(started.ok, true);
const id = started.sessionId;
assert.ok(id);
assert.equal(controller.status().microphone.state, "on");
+ assert.equal(controller.status().microphone.activeDevice?.id, "usb-desk");
+ assert.deepEqual(audio.deviceIds, ["usb-desk"]);
const discarded = await controller.discard();
assert.equal(discarded.ok, true);
@@ -185,6 +198,7 @@ test("unexpected microphone termination updates live status without ending video
assert.deepEqual(controller.status().microphone, {
state: "error",
error: "The microphone disconnected.",
+ activeDevice: null,
});
assert.equal((await controller.stop()).ok, true);
});
@@ -250,6 +264,31 @@ test("microphone failure is surfaced without stopping screen capture", async ()
});
});
+test("switching an active microphone closes one segment before starting the next", async () => {
+ await withSessionsRoot(async () => {
+ const audio = new FakeAudioRecorder();
+ const controller = new RecorderController({
+ resolveConfig: () => ({ ...FULL_CAPTURE, video: false }),
+ buildCollectors: () => [],
+ createAudioRecorder: () => audio,
+ deleteSession: async () => undefined,
+ });
+
+ assert.equal((await controller.start()).ok, true);
+ assert.equal(
+ (await controller.setMicrophoneEnabled(true, "built-in")).ok,
+ true,
+ );
+ assert.equal((await controller.setMicrophoneDevice("usb-desk")).ok, true);
+ assert.deepEqual(audio.calls, ["start", "enable", "disable", "enable"]);
+ assert.deepEqual(audio.deviceIds, ["built-in", "usb-desk"]);
+ assert.equal(controller.status().microphone.state, "on");
+ assert.equal(controller.status().microphone.activeDevice?.id, "usb-desk");
+
+ assert.equal((await controller.stop()).ok, true);
+ });
+});
+
test("stop waits behind an in-flight microphone enable", async () => {
await withSessionsRoot(async () => {
let releaseEnable: () => void = () => undefined;
diff --git a/electron/recorder/controller.ts b/electron/recorder/controller.ts
index 6b7d315..88df385 100644
--- a/electron/recorder/controller.ts
+++ b/electron/recorder/controller.ts
@@ -14,6 +14,10 @@ import type {
StopResult,
} from "../../common/ipc";
import type { RecorderState, SessionMeta } from "../../common/types";
+import {
+ SYSTEM_DEFAULT_MICROPHONE_ID,
+ type MicrophoneDevice,
+} from "../../common/microphone";
import { createLogger } from "../logger";
import type { Collector } from "./collector";
import { CollectorHost } from "./collector";
@@ -39,7 +43,7 @@ export interface SessionVideoRecorder {
/** Microphone sidecar that can create multiple capture intervals per session. */
export interface SessionAudioRecorder {
start(sessionDir: string, sessionStartedAt: number): Promise<void>;
- enable(): Promise<void>;
+ enable(deviceId?: string): Promise<MicrophoneDevice | void>;
disable(): Promise<unknown>;
finish(videoStartEpoch: number | null): Promise<unknown>;
}
@@ -81,7 +85,11 @@ export class RecorderController {
private video: SessionVideoRecorder | null = null;
private audio: SessionAudioRecorder | null = null;
private transition: RecorderStatus["transition"] = "none";
- private microphone: RecorderStatus["microphone"] = { state: "off", error: null };
+ private microphone: RecorderStatus["microphone"] = {
+ state: "off",
+ error: null,
+ activeDevice: null,
+ };
private lastFinish: RecorderStatus["lastFinish"] = null;
private shuttingDown = false;
private operations: Promise<void> = Promise.resolve();
@@ -124,7 +132,12 @@ export class RecorderController {
startedAt: this.store?.meta.startedAt ?? null,
eventCount: this.store?.eventCount ?? 0,
transition: this.transition,
- microphone: { ...this.microphone },
+ microphone: {
+ ...this.microphone,
+ activeDevice: this.microphone.activeDevice
+ ? { ...this.microphone.activeDevice }
+ : null,
+ },
lastFinish: this.lastFinish ? { ...this.lastFinish } : null,
lastSession: this.lastCompleted
? { id: this.lastCompleted.id, processed: this.lastProcessed }
@@ -155,12 +168,65 @@ export class RecorderController {
return this.enqueue(() => this.finish("discard"));
}
- setMicrophoneEnabled(enabled: boolean): Promise<MicrophoneResult> {
+ setMicrophoneEnabled(
+ enabled: boolean,
+ deviceId = SYSTEM_DEFAULT_MICROPHONE_ID,
+ ): Promise<MicrophoneResult> {
return this.enqueue(async () => {
if (!this.store || this.transition !== "none") {
return { ok: false, error: "Microphone controls are only available while recording." };
}
- return this.setMicrophoneInternal(enabled);
+ return this.setMicrophoneInternal(enabled, deviceId);
+ });
+ }
+
+ setMicrophoneDevice(deviceId: string): Promise<MicrophoneResult> {
+ return this.enqueue(async () => {
+ if (!this.store || this.transition !== "none") {
+ return { ok: false, error: "Microphone controls are only available while recording." };
+ }
+ if (this.microphone.state !== "on") {
+ return { ok: true, state: this.microphone.state };
+ }
+ if (!this.audio) {
+ const error = "Microphone capture is unavailable for this recording.";
+ this.microphone = { state: "error", error, activeDevice: null };
+ this.emit();
+ return { ok: false, state: "error", error };
+ }
+
+ this.microphone = {
+ state: "stopping",
+ error: null,
+ activeDevice: this.microphone.activeDevice,
+ };
+ this.emit();
+ try {
+ await this.audio.disable();
+ } catch (error) {
+ const message = error instanceof Error ? error.message : String(error);
+ this.microphone = { state: "error", error: message, activeDevice: null };
+ this.emit();
+ return { ok: false, state: "error", error: message };
+ }
+
+ this.microphone = { state: "starting", error: null, activeDevice: null };
+ this.emit();
+ try {
+ const result = await this.audio.enable(deviceId);
+ this.microphone = {
+ state: "on",
+ error: null,
+ activeDevice: isMicrophoneDevice(result) ? result : null,
+ };
+ this.emit();
+ return { ok: true, state: "on" };
+ } catch (error) {
+ const message = error instanceof Error ? error.message : String(error);
+ this.microphone = { state: "error", error: message, activeDevice: null };
+ this.emit();
+ return { ok: false, state: "error", error: message };
+ }
});
}
@@ -188,7 +254,7 @@ export class RecorderController {
if (this.store) return { ok: false, error: "Already recording" };
this.transition = "starting";
- this.microphone = { state: "off", error: null };
+ this.microphone = { state: "off", error: null, activeDevice: null };
this.lastFinish = null;
this.emit();
@@ -246,12 +312,15 @@ export class RecorderController {
const message = error instanceof Error ? error.message : String(error);
log.warn("microphone initialization failed:", message);
this.audio = null;
- this.microphone = { state: "error", error: message };
+ this.microphone = { state: "error", error: message, activeDevice: null };
}
}
if (options?.narration && this.audio) {
- await this.setMicrophoneInternal(true);
+ await this.setMicrophoneInternal(
+ true,
+ options.microphoneDeviceId ?? SYSTEM_DEFAULT_MICROPHONE_ID,
+ );
}
this.transition = "none";
@@ -259,26 +328,33 @@ export class RecorderController {
return { ok: true, sessionId: store.meta.id };
}
- private async setMicrophoneInternal(enabled: boolean): Promise<MicrophoneResult> {
+ private async setMicrophoneInternal(
+ enabled: boolean,
+ deviceId = SYSTEM_DEFAULT_MICROPHONE_ID,
+ ): Promise<MicrophoneResult> {
if (enabled) {
if (this.microphone.state === "on") return { ok: true, state: "on" };
if (!this.audio) {
const error = "Microphone capture is unavailable for this recording.";
- this.microphone = { state: "error", error };
+ this.microphone = { state: "error", error, activeDevice: null };
this.emit();
return { ok: false, state: "error", error };
}
- this.microphone = { state: "starting", error: null };
+ this.microphone = { state: "starting", error: null, activeDevice: null };
this.emit();
try {
- await this.audio.enable();
- this.microphone = { state: "on", error: null };
+ const result = await this.audio.enable(deviceId);
+ this.microphone = {
+ state: "on",
+ error: null,
+ activeDevice: isMicrophoneDevice(result) ? result : null,
+ };
this.emit();
return { ok: true, state: "on" };
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
- this.microphone = { state: "error", error: message };
+ this.microphone = { state: "error", error: message, activeDevice: null };
this.emit();
return { ok: false, state: "error", error: message };
}
@@ -286,21 +362,25 @@ export class RecorderController {
if (this.microphone.state === "off") return { ok: true, state: "off" };
if (!this.audio) {
- this.microphone = { state: "off", error: null };
+ this.microphone = { state: "off", error: null, activeDevice: null };
this.emit();
return { ok: true, state: "off" };
}
- this.microphone = { state: "stopping", error: null };
+ this.microphone = {
+ state: "stopping",
+ error: null,
+ activeDevice: this.microphone.activeDevice,
+ };
this.emit();
try {
await this.audio.disable();
- this.microphone = { state: "off", error: null };
+ this.microphone = { state: "off", error: null, activeDevice: null };
this.emit();
return { ok: true, state: "off" };
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
- this.microphone = { state: "error", error: message };
+ this.microphone = { state: "error", error: message, activeDevice: null };
this.emit();
return { ok: false, state: "error", error: message };
}
@@ -312,7 +392,11 @@ export class RecorderController {
this.transition = intent === "save" ? "stopping" : "discarding";
if (this.microphone.state !== "off") {
- this.microphone = { state: "stopping", error: null };
+ this.microphone = {
+ state: "stopping",
+ error: null,
+ activeDevice: this.microphone.activeDevice,
+ };
}
this.emit();
@@ -351,7 +435,7 @@ export class RecorderController {
this.bus.detach();
store.finalize(Date.now());
await store.whenClosed();
- this.microphone = { state: "off", error: null };
+ this.microphone = { state: "off", error: null, activeDevice: null };
if (intent === "discard") {
try {
@@ -427,7 +511,7 @@ export class RecorderController {
private onAudioCaptureEnded(event: AudioCaptureEnded): void {
if (!this.store || this.transition !== "none") return;
const error = event.error ?? "The microphone stopped unexpectedly.";
- this.microphone = { state: "error", error };
+ this.microphone = { state: "error", error, activeDevice: null };
this.emit();
}
@@ -446,3 +530,16 @@ function readStartEpoch(value: unknown): number | null {
const startEpoch = value.startEpoch;
return typeof startEpoch === "number" && Number.isFinite(startEpoch) ? startEpoch : null;
}
+
+function isMicrophoneDevice(value: unknown): value is MicrophoneDevice {
+ return (
+ !!value &&
+ typeof value === "object" &&
+ "id" in value &&
+ typeof value.id === "string" &&
+ "label" in value &&
+ typeof value.label === "string" &&
+ "groupId" in value &&
+ typeof value.groupId === "string"
+ );
+}
diff --git a/electron/recording-controls-bounds.test.ts b/electron/recording-controls-bounds.test.ts
index 8eee7d3..4f3f5c4 100644
--- a/electron/recording-controls-bounds.test.ts
+++ b/electron/recording-controls-bounds.test.ts
@@ -36,7 +36,7 @@ test("recording controls clamp to negative-coordinate and small displays", () =>
);
});
-test("confirmation expansion preserves the bar's bottom edge", () => {
+test("overlay panel expansion preserves the bar's bottom edge", () => {
const workArea = { x: 0, y: 0, width: 1440, height: 900 };
const collapsed = { x: 486, y: 806, width: 468, height: 76 };
const expanded = resizeRecordingControlsBounds(collapsed, true, workArea);
@@ -47,4 +47,3 @@ test("confirmation expansion preserves the bar's bottom edge", () => {
collapsed,
);
});
-
diff --git a/electron/recording-controls-bounds.ts b/electron/recording-controls-bounds.ts
index 66a4913..a878527 100644
--- a/electron/recording-controls-bounds.ts
+++ b/electron/recording-controls-bounds.ts
@@ -45,7 +45,7 @@ export function clampRecordingControlsBounds(
/**
* Resize the transparent overlay above its fixed bar. Its bottom edge remains
- * stationary, so opening confirmation never makes the control bar jump.
+ * stationary, so opening a menu or confirmation never makes the control bar jump.
*/
export function resizeRecordingControlsBounds(
bounds: WindowBounds,
@@ -71,4 +71,3 @@ function clamp(value: number, min: number, max: number): number {
if (max < min) return min;
return Math.max(min, Math.min(max, value));
}
-
diff --git a/electron/tray.ts b/electron/tray.ts
index 1e80f2c..c3a54be 100644
--- a/electron/tray.ts
+++ b/electron/tray.ts
@@ -23,6 +23,7 @@ function trayIcon(): Electron.NativeImage {
export function createTray(
recorder: RecorderController,
+ startRecording: () => Promise<unknown>,
showRecorderWindow: () => void,
showRecordingControls: () => void,
): Tray {
@@ -35,7 +36,7 @@ export function createTray(
Menu.buildFromTemplate([
{
label: recording ? "Stop recording" : "Start recording",
- click: () => void (recording ? recorder.stop() : recorder.start()),
+ click: () => void (recording ? recorder.stop() : startRecording()),
},
{ type: "separator" },
{
diff --git a/electron/window.ts b/electron/window.ts
index 129fdf0..edf9b4a 100644
--- a/electron/window.ts
+++ b/electron/window.ts
@@ -116,7 +116,7 @@ export function createRecordingControlsWindow(): BrowserWindow {
return win;
}
-/** Resize confirmation above the bar while preserving its on-screen bottom edge. */
+/** Resize a panel above the bar while preserving its on-screen bottom edge. */
export function setRecordingControlsExpanded(
win: BrowserWindow,
expanded: boolean,
diff --git a/package.json b/package.json
index ab18bd9..30eeeaf 100644
--- a/package.json
+++ b/package.json
@@ -14,7 +14,7 @@
"build": "tsc --noEmit && vite build",
"typecheck": "tsc --noEmit",
"typecheck:evals": "tsc --noEmit -p evals/tsconfig.json",
- "test": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs --test common/audio.test.ts electron/recording-controls-bounds.test.ts electron/recorder/controller.test.ts electron/frames/extractor.test.ts electron/narration/audio-analysis.test.ts electron/sessions.test.ts",
+ "test": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs --test common/audio.test.ts common/microphone.test.ts electron/recording-controls-bounds.test.ts electron/recorder/controller.test.ts electron/frames/extractor.test.ts electron/narration/audio-analysis.test.ts electron/sessions.test.ts",
"eval": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs evals/run.ts",
"eval:builder": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs evals/builder/run.ts",
"eval:skill": "node --experimental-transform-types --no-warnings --import ./evals/register.mjs evals/skillbuilder/run.ts",
diff --git a/src/App.css b/src/App.css
index e8004bf..619eb1b 100644
--- a/src/App.css
+++ b/src/App.css
@@ -444,20 +444,28 @@ button:focus-visible,
}
.narrate {
- display: flex;
- align-items: center;
- gap: 12px;
+ flex: none;
width: 100%;
- text-align: left;
background: transparent;
border: 1px solid var(--line);
color: var(--text);
border-radius: 12px;
- padding: 11px 13px;
+ overflow: hidden;
transition: border-color 0.15s ease, background 0.15s ease;
}
-.narrate:not(:disabled):hover {
+.narrate-toggle {
+ display: flex;
+ align-items: center;
+ gap: 12px;
+ width: 100%;
+ padding: 11px 13px;
+ text-align: left;
+ background: transparent;
+ color: inherit;
+}
+
+.narrate-toggle:not(:disabled):hover {
border-color: var(--line-strong);
background: var(--surface);
}
@@ -467,7 +475,7 @@ button:focus-visible,
background: var(--signal-dim);
}
-.narrate:disabled {
+.narrate-toggle:disabled {
opacity: 0.9;
cursor: default;
}
@@ -510,11 +518,11 @@ button:focus-visible,
.narrate-switch {
flex: none;
box-sizing: border-box;
- width: 34px;
- height: 20px;
+ width: 36px;
+ height: 22px;
border-radius: 999px;
background: var(--line-strong);
- padding: 2px;
+ padding: 3px;
transition: background 0.15s ease;
}
@@ -536,6 +544,71 @@ button:focus-visible,
transform: translateX(14px);
}
+.narrate-device {
+ display: grid;
+ grid-template-columns: auto minmax(0, 1fr);
+ align-items: center;
+ gap: 6px 10px;
+ padding: 6px 13px 6px 43px;
+ border-top: 1px solid rgba(var(--brand-rgb), 0.14);
+}
+
+.narrate-device label {
+ font-size: 10.5px;
+ font-weight: 600;
+ color: var(--brand-ink);
+}
+
+.narrate-select-wrap {
+ position: relative;
+ min-width: 0;
+}
+
+.narrate-select-wrap select {
+ width: 100%;
+ height: 30px;
+ padding: 0 27px 0 9px;
+ appearance: none;
+ border: 1px solid rgba(var(--brand-rgb), 0.34);
+ border-radius: 8px;
+ background: var(--surface);
+ color: var(--text);
+ font: inherit;
+ font-size: 11px;
+ text-overflow: ellipsis;
+}
+
+.narrate-select-wrap select:focus-visible {
+ outline: 2px solid rgba(var(--brand-rgb), 0.5);
+ outline-offset: 1px;
+}
+
+.narrate-select-chevron {
+ position: absolute;
+ top: 50%;
+ right: 9px;
+ color: var(--muted);
+ font-size: 10px;
+ pointer-events: none;
+ transform: translateY(-52%);
+}
+
+.narrate-device-note {
+ grid-column: 1 / -1;
+ margin: 0;
+ color: var(--muted);
+ font-size: 10px;
+ line-height: 1.35;
+}
+
+.narrate-device-note.warn {
+ color: #8a5d00;
+}
+
+.narrate-device-note.error {
+ color: var(--rec);
+}
+
.privacy-note {
display: flex;
align-items: center;
@@ -778,7 +851,9 @@ button:focus-visible,
.recording-bar button,
.recording-discard-confirm,
- .recording-discard-confirm button {
+ .recording-discard-confirm button,
+ .recording-microphone-menu,
+ .recording-microphone-menu button {
-webkit-app-region: no-drag;
}
@@ -827,33 +902,54 @@ button:focus-visible,
background: rgba(255, 255, 255, 0.13);
}
- .recording-microphone {
+ .recording-microphone-split {
display: flex;
align-items: center;
- justify-content: center;
- gap: 5px;
- min-width: 67px;
height: 36px;
- padding: 0 8px;
border: 1px solid rgba(255, 255, 255, 0.14);
border-radius: 9px;
background: #2d2d30;
color: #b8b8bc;
+ overflow: hidden;
+ -webkit-app-region: no-drag;
+ }
+
+ .recording-microphone {
+ display: flex;
+ align-items: center;
+ justify-content: center;
+ gap: 5px;
+ min-width: 59px;
+ height: 34px;
+ padding: 0 7px;
+ background: transparent;
+ color: inherit;
font-size: 10px;
}
- .recording-microphone.on {
+ .recording-microphone-menu-toggle {
+ display: grid;
+ place-items: center;
+ width: 23px;
+ height: 34px;
+ border-left: 1px solid rgba(255, 255, 255, 0.12);
+ background: transparent;
+ color: inherit;
+ }
+
+ .recording-microphone-split.on {
border-color: rgba(239, 89, 75, 0.5);
background: rgba(239, 89, 75, 0.15);
color: #ff8a80;
}
- .recording-microphone.error {
+ .recording-microphone-split.error {
border-color: rgba(255, 184, 77, 0.55);
color: #ffc269;
}
.recording-microphone:not(:disabled):hover,
+ .recording-microphone-menu-toggle:not(:disabled):hover,
.recording-discard:not(:disabled):hover {
filter: brightness(1.16);
}
@@ -890,7 +986,8 @@ button:focus-visible,
}
.recording-bar button:focus-visible,
- .recording-discard-confirm button:focus-visible {
+ .recording-discard-confirm button:focus-visible,
+ .recording-microphone-menu button:focus-visible {
outline: 2px solid #ff766a;
outline-offset: 2px;
}
@@ -918,6 +1015,95 @@ button:focus-visible,
animation: recording-confirm-in 0.15s ease-out;
}
+ .recording-microphone-menu {
+ display: flex;
+ flex-direction: column;
+ gap: 9px;
+ width: 100%;
+ max-height: 150px;
+ padding: 12px;
+ border: 1px solid rgba(255, 255, 255, 0.14);
+ border-radius: 13px;
+ background: rgba(37, 37, 40, 0.98);
+ box-shadow: 0 14px 34px rgba(0, 0, 0, 0.42);
+ color: #f7f7f7;
+ animation: recording-confirm-in 0.15s ease-out;
+ }
+
+ .recording-microphone-menu header {
+ display: flex;
+ align-items: baseline;
+ gap: 8px;
+ min-width: 0;
+ }
+
+ .recording-microphone-menu header strong {
+ flex: none;
+ font-size: 12px;
+ }
+
+ .recording-microphone-menu header span {
+ overflow: hidden;
+ color: #aaa;
+ font-size: 10px;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+ }
+
+ .recording-microphone-options {
+ display: flex;
+ flex-direction: column;
+ gap: 3px;
+ min-height: 0;
+ overflow-y: auto;
+ }
+
+ .recording-microphone-options button {
+ display: flex;
+ align-items: center;
+ justify-content: space-between;
+ gap: 12px;
+ min-height: 29px;
+ padding: 0 8px;
+ border-radius: 7px;
+ background: transparent;
+ color: #e6e6e8;
+ font-size: 10.5px;
+ text-align: left;
+ }
+
+ .recording-microphone-options button:hover,
+ .recording-microphone-options button.selected {
+ background: rgba(255, 255, 255, 0.08);
+ }
+
+ .recording-microphone-options button > span:first-child {
+ overflow: hidden;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+ }
+
+ .recording-microphone-selected {
+ flex: none;
+ color: #ff8a80;
+ font-size: 9px;
+ }
+
+ .recording-microphone-menu p {
+ margin: 0;
+ color: #c7c7ca;
+ font-size: 10px;
+ line-height: 1.35;
+ }
+
+ .recording-microphone-menu p.warn {
+ color: #ffc269;
+ }
+
+ .recording-microphone-menu p.error {
+ color: #ff8a80;
+ }
+
.recording-discard-confirm h2 {
margin: 0 0 4px;
font-size: 13px;
@@ -979,7 +1165,8 @@ button:focus-visible,
@media (prefers-reduced-motion: reduce) {
.recording-live-dot,
- .recording-discard-confirm {
+ .recording-discard-confirm,
+ .recording-microphone-menu {
animation: none;
}
}
diff --git a/src/Recorder.tsx b/src/Recorder.tsx
index eaef7f9..b0c7e69 100644
--- a/src/Recorder.tsx
+++ b/src/Recorder.tsx
@@ -1,6 +1,11 @@
import { useCallback, useEffect, useState } from "react";
-import type { DoctorReport, NarrationStatus, RecorderStatus } from "../common/ipc";
+import type {
+ DoctorReport,
+ MicrophoneSettingsStatus,
+ NarrationStatus,
+ RecorderStatus,
+} from "../common/ipc";
import { formatMs } from "./format";
import { WhatsRecorded } from "./WhatsRecorded";
@@ -12,8 +17,11 @@ export function Recorder() {
const [status, setStatus] = useState<RecorderStatus | null>(null);
const [doctor, setDoctor] = useState<DoctorReport | null>(null);
const [narrationStatus, setNarrationStatus] = useState<NarrationStatus | null>(null);
+ const [microphoneSettings, setMicrophoneSettings] =
+ useState<MicrophoneSettingsStatus | null>(null);
const [showPrivacy, setShowPrivacy] = useState(false);
- const [narrate, setNarrate] = useState(false);
+ const [microphonePending, setMicrophonePending] = useState(false);
+ const [microphoneActionError, setMicrophoneActionError] = useState<string | null>(null);
const [elapsed, setElapsed] = useState(0);
const [sessionCount, setSessionCount] = useState(0);
const [pendingCount, setPendingCount] = useState(0);
@@ -28,12 +36,16 @@ export function Recorder() {
void window.skillRecorder.status().then(setStatus);
void window.skillRecorder.doctor().then(setDoctor);
void window.skillRecorder.narrationStatus().then(setNarrationStatus);
+ void window.skillRecorder.microphoneSettings().then(setMicrophoneSettings);
void refreshCount();
const offRecorder = window.skillRecorder.onStatusChanged(setStatus);
const offNarration = window.skillRecorder.onNarrationStatusChanged(setNarrationStatus);
+ const offMicrophones =
+ window.skillRecorder.onMicrophoneSettingsChanged(setMicrophoneSettings);
return () => {
offRecorder();
offNarration();
+ offMicrophones();
};
}, [refreshCount]);
@@ -50,6 +62,7 @@ export function Recorder() {
const startedAt = status?.startedAt ?? null;
const justSaved = !recording && status?.lastFinish?.outcome === "saved";
const justDiscarded = !recording && status?.lastFinish?.outcome === "discarded";
+ const narrate = microphoneSettings?.narrationEnabled ?? false;
// Refresh the library count whenever a recording finishes.
useEffect(() => {
@@ -68,10 +81,39 @@ export function Recorder() {
const toggle = useCallback(async () => {
const res = recording
? await window.skillRecorder.stop()
- : await window.skillRecorder.start({ narration: narrate });
+ : await window.skillRecorder.start();
if (!res.ok) window.alert(res.error ?? "Action failed");
setStatus(await window.skillRecorder.status());
- }, [recording, narrate]);
+ }, [recording]);
+
+ const toggleNarration = useCallback(async () => {
+ if (!microphoneSettings) return;
+ setMicrophonePending(true);
+ setMicrophoneActionError(null);
+ const result = await window.skillRecorder.setNarrationEnabled(
+ !microphoneSettings.narrationEnabled,
+ );
+ setMicrophoneSettings(result.status);
+ if (!result.ok) {
+ setMicrophoneActionError(
+ result.error ?? "Could not update the narration preference.",
+ );
+ }
+ setMicrophonePending(false);
+ }, [microphoneSettings]);
+
+ const selectMicrophone = useCallback(async (deviceId: string) => {
+ setMicrophonePending(true);
+ setMicrophoneActionError(null);
+ const result = await window.skillRecorder.selectMicrophone(deviceId);
+ setMicrophoneSettings(result.status);
+ if (!result.ok) {
+ setMicrophoneActionError(
+ result.error ?? "Could not select that microphone.",
+ );
+ }
+ setMicrophonePending(false);
+ }, []);
const openLibrary = useCallback(() => {
void window.skillRecorder.openLibrary();
@@ -99,7 +141,7 @@ export function Recorder() {
<button
className={`record ${recording ? "on" : ""}`}
onClick={toggle}
- disabled={transitioning}
+ disabled={transitioning || microphonePending}
aria-label={recording ? "Stop recording" : "Start recording"}
>
<span className="record-glyph" />
@@ -118,43 +160,109 @@ export function Recorder() {
</div>
</div>
- <button
- className={`narrate ${narrate ? "on" : ""}`}
- role="switch"
- aria-checked={narrate}
- onClick={() => setNarrate((v) => !v)}
- disabled={recording || transitioning}
- >
- <span className="narrate-icon" aria-hidden>
- <svg width="18" height="18" viewBox="0 0 20 20" fill="none">
- <rect x="7.5" y="2.5" width="5" height="9" rx="2.5" stroke="currentColor" strokeWidth="1.4" />
- <path
- d="M5 9.2a5 5 0 0 0 10 0"
- stroke="currentColor"
- strokeWidth="1.4"
- strokeLinecap="round"
- />
- <path d="M10 14.2v3" stroke="currentColor" strokeWidth="1.4" strokeLinecap="round" />
- </svg>
- </span>
- <span className="narrate-text">
- <span className="narrate-label">Narrate</span>
- <span className="narrate-sub">
- {recording
- ? narrate
- ? "Listening to your voice"
- : "Voice off for this recording"
- : narrate
- ? narrationStatus?.model === "ready"
- ? "Your voice will be included"
- : "Voice saved · transcription needs ~250 MB later"
- : "Explain out loud (optional)"}
+ <section className={`narrate ${narrate ? "on" : ""}`}>
+ <button
+ className="narrate-toggle"
+ role="switch"
+ aria-checked={narrate}
+ aria-busy={microphonePending}
+ onClick={() => void toggleNarration()}
+ disabled={
+ !microphoneSettings ||
+ microphonePending ||
+ recording ||
+ transitioning
+ }
+ >
+ <span className="narrate-icon" aria-hidden>
+ <svg width="18" height="18" viewBox="0 0 20 20" fill="none">
+ <rect
+ x="7.5"
+ y="2.5"
+ width="5"
+ height="9"
+ rx="2.5"
+ stroke="currentColor"
+ strokeWidth="1.4"
+ />
+ <path
+ d="M5 9.2a5 5 0 0 0 10 0"
+ stroke="currentColor"
+ strokeWidth="1.4"
+ strokeLinecap="round"
+ />
+ <path
+ d="M10 14.2v3"
+ stroke="currentColor"
+ strokeWidth="1.4"
+ strokeLinecap="round"
+ />
+ </svg>
</span>
- </span>
- <span className={`narrate-switch ${narrate ? "on" : ""}`} aria-hidden>
- <span className="narrate-knob" />
- </span>
- </button>
+ <span className="narrate-text">
+ <span className="narrate-label">Narrate</span>
+ <span className="narrate-sub">
+ {microphonePending
+ ? "Requesting microphone access..."
+ : microphoneActionError || microphoneSettings?.error
+ ? "Microphone needs attention"
+ : recording
+ ? narrate
+ ? "Listening to your voice"
+ : "Voice off for this recording"
+ : narrate
+ ? `Will use ${microphoneSettings?.selectedDeviceLabel ?? "System default"}`
+ : "Explain out loud (optional)"}
+ </span>
+ </span>
+ <span className={`narrate-switch ${narrate ? "on" : ""}`} aria-hidden>
+ <span className="narrate-knob" />
+ </span>
+ </button>
+
+ {narrate && microphoneSettings && (
+ <div className="narrate-device">
+ <label htmlFor="narrate-microphone">Microphone</label>
+ <div className="narrate-select-wrap">
+ <select
+ id="narrate-microphone"
+ value={microphoneSettings.selectedDeviceId}
+ disabled={microphonePending || recording || transitioning}
+ onChange={(event) => void selectMicrophone(event.target.value)}
+ >
+ {microphoneSettings.devices.map((device) => (
+ <option key={device.id} value={device.id}>
+ {device.label}
+ </option>
+ ))}
+ </select>
+ <span className="narrate-select-chevron" aria-hidden>
+ ▾
+ </span>
+ </div>
+ {(microphoneActionError ||
+ microphoneSettings.error ||
+ microphoneSettings.fallback) && (
+ <p
+ className={`narrate-device-note ${
+ microphoneActionError || microphoneSettings.error
+ ? "error"
+ : "warn"
+ }`}
+ role={
+ microphoneActionError || microphoneSettings.error
+ ? "alert"
+ : undefined
+ }
+ >
+ {microphoneActionError ??
+ microphoneSettings.error ??
+ microphoneSettings.fallback}
+ </p>
+ )}
+ </div>
+ )}
+ </section>
<button className="privacy-note" onClick={() => setShowPrivacy(true)}>
<span className="privacy-note-icon" aria-hidden>
diff --git a/src/RecordingControls.tsx b/src/RecordingControls.tsx
index e2c09f1..85c9953 100644
--- a/src/RecordingControls.tsx
+++ b/src/RecordingControls.tsx
@@ -1,20 +1,36 @@
import { useCallback, useEffect, useRef, useState } from "react";
-import type { RecorderStatus } from "../common/ipc";
+import type {
+ MicrophoneSettingsStatus,
+ RecorderStatus,
+} from "../common/ipc";
import { formatMs } from "./format";
export function RecordingControls() {
const [status, setStatus] = useState<RecorderStatus | null>(null);
+ const [microphoneSettings, setMicrophoneSettings] =
+ useState<MicrophoneSettingsStatus | null>(null);
const [elapsed, setElapsed] = useState(0);
const [confirmDiscard, setConfirmDiscard] = useState(false);
+ const [showMicrophoneMenu, setShowMicrophoneMenu] = useState(false);
const [microphonePending, setMicrophonePending] = useState(false);
+ const [devicePending, setDevicePending] = useState(false);
const [finishPending, setFinishPending] = useState<"done" | "discard" | null>(null);
const [actionError, setActionError] = useState<string | null>(null);
const keepRecordingRef = useRef<HTMLButtonElement>(null);
+ const microphoneControlRef = useRef<HTMLDivElement>(null);
+ const microphoneMenuRef = useRef<HTMLElement>(null);
useEffect(() => {
void window.skillRecorder.status().then(setStatus);
- return window.skillRecorder.onStatusChanged(setStatus);
+ void window.skillRecorder.microphoneSettings().then(setMicrophoneSettings);
+ const offStatus = window.skillRecorder.onStatusChanged(setStatus);
+ const offMicrophones =
+ window.skillRecorder.onMicrophoneSettingsChanged(setMicrophoneSettings);
+ return () => {
+ offStatus();
+ offMicrophones();
+ };
}, []);
const recording = status?.state === "recording";
@@ -33,13 +49,17 @@ export function RecordingControls() {
useEffect(() => {
if (recording) return;
if (confirmDiscard) setConfirmDiscard(false);
+ if (showMicrophoneMenu) setShowMicrophoneMenu(false);
setMicrophonePending(false);
+ setDevicePending(false);
setFinishPending(null);
- }, [confirmDiscard, recording]);
+ }, [confirmDiscard, recording, showMicrophoneMenu]);
useEffect(() => {
- void window.skillRecorder.setRecordingControlsExpanded(confirmDiscard);
- }, [confirmDiscard]);
+ void window.skillRecorder.setRecordingControlsExpanded(
+ confirmDiscard || showMicrophoneMenu,
+ );
+ }, [confirmDiscard, showMicrophoneMenu]);
useEffect(() => {
if (!confirmDiscard) return;
@@ -51,6 +71,48 @@ export function RecordingControls() {
return () => window.removeEventListener("keydown", onKeyDown);
}, [confirmDiscard]);
+ useEffect(() => {
+ if (!showMicrophoneMenu) return;
+ microphoneMenuRef.current
+ ?.querySelector<HTMLButtonElement>('[aria-pressed="true"]')
+ ?.focus({ preventScroll: true });
+ const onKeyDown = (event: KeyboardEvent) => {
+ if (event.key === "Escape") setShowMicrophoneMenu(false);
+ };
+ const onMouseDown = (event: MouseEvent) => {
+ const target = event.target;
+ if (!(target instanceof Node)) return;
+ if (
+ microphoneMenuRef.current?.contains(target) ||
+ microphoneControlRef.current?.contains(target)
+ ) {
+ return;
+ }
+ setShowMicrophoneMenu(false);
+ };
+ window.addEventListener("keydown", onKeyDown);
+ window.addEventListener("mousedown", onMouseDown);
+ return () => {
+ window.removeEventListener("keydown", onKeyDown);
+ window.removeEventListener("mousedown", onMouseDown);
+ };
+ }, [showMicrophoneMenu]);
+
+ useEffect(() => {
+ if (
+ recording &&
+ (status?.microphone.state === "error" ||
+ microphoneSettings?.preferredDeviceUnavailable)
+ ) {
+ setConfirmDiscard(false);
+ setShowMicrophoneMenu(true);
+ }
+ }, [
+ microphoneSettings?.preferredDeviceUnavailable,
+ recording,
+ status?.microphone.state,
+ ]);
+
const toggleMicrophone = useCallback(async () => {
if (!status) return;
setMicrophonePending(true);
@@ -61,6 +123,19 @@ export function RecordingControls() {
setMicrophonePending(false);
}, [status]);
+ const selectMicrophone = useCallback(async (deviceId: string) => {
+ setDevicePending(true);
+ setActionError(null);
+ const result = await window.skillRecorder.selectMicrophone(deviceId);
+ setMicrophoneSettings(result.status);
+ if (!result.ok) {
+ setActionError(result.error ?? "Could not switch microphones.");
+ } else {
+ setShowMicrophoneMenu(false);
+ }
+ setDevicePending(false);
+ }, []);
+
const done = useCallback(async () => {
setFinishPending("done");
setActionError(null);
@@ -86,11 +161,16 @@ export function RecordingControls() {
const transitionBusy = status?.transition !== "none";
const microphoneBusy =
microphonePending ||
+ devicePending ||
status?.microphone.state === "starting" ||
status?.microphone.state === "stopping";
const lifecycleBusy = finishPending !== null || transitionBusy || !recording;
const microphoneOn = status?.microphone.state === "on";
const microphoneError = status?.microphone.state === "error";
+ const activeMicrophoneLabel =
+ status?.microphone.activeDevice?.label ??
+ microphoneSettings?.selectedDeviceLabel ??
+ "System default";
const microphoneLabel =
status?.microphone.state === "starting"
? "Starting"
@@ -101,7 +181,12 @@ export function RecordingControls() {
: microphoneError
? "Retry"
: "Off";
- const error = actionError ?? status?.microphone.error ?? null;
+ const error =
+ actionError ??
+ status?.microphone.error ??
+ microphoneSettings?.error ??
+ microphoneSettings?.fallback ??
+ null;
const captureLabel =
status?.transition === "starting"
? "Starting"
@@ -152,6 +237,71 @@ export function RecordingControls() {
</section>
)}
+ {showMicrophoneMenu && (
+ <section
+ ref={microphoneMenuRef}
+ className="recording-microphone-menu"
+ aria-label="Choose microphone"
+ >
+ <header>
+ <strong>Microphone</strong>
+ <span>
+ {microphoneOn
+ ? `Using ${activeMicrophoneLabel}`
+ : `Next: ${microphoneSettings?.selectedDeviceLabel ?? "System default"}`}
+ </span>
+ </header>
+ <div
+ className="recording-microphone-options"
+ role="radiogroup"
+ aria-label="Audio input"
+ >
+ {microphoneSettings?.devices.map((device) => {
+ const selected =
+ device.id === microphoneSettings.selectedDeviceId;
+ return (
+ <button
+ key={device.id}
+ className={selected ? "selected" : ""}
+ role="radio"
+ aria-checked={selected}
+ aria-pressed={selected}
+ disabled={devicePending || finishPending !== null}
+ onClick={() => void selectMicrophone(device.id)}
+ >
+ <span>{device.label}</span>
+ {selected && (
+ <span className="recording-microphone-selected">
+ Selected
+ </span>
+ )}
+ </button>
+ );
+ })}
+ </div>
+ {error && (
+ <p
+ className={
+ actionError ||
+ status?.microphone.error ||
+ microphoneSettings?.error
+ ? "error"
+ : "warn"
+ }
+ role={
+ actionError ||
+ status?.microphone.error ||
+ microphoneSettings?.error
+ ? "alert"
+ : undefined
+ }
+ >
+ {error}
+ </p>
+ )}
+ </section>
+ )}
+
<div className="recording-bar-wrap">
<div className="recording-bar">
<div className="recording-live" role="status" aria-label={`${captureLabel} ${formatMs(elapsed)}`}>
@@ -164,31 +314,56 @@ export function RecordingControls() {
<span className="recording-divider" aria-hidden />
- <button
- className={`recording-microphone ${microphoneOn ? "on" : ""} ${
- microphoneError ? "error" : ""
- }`}
- disabled={lifecycleBusy || microphoneBusy}
- aria-label={
- microphoneOn
- ? "Turn microphone off"
- : microphoneError
- ? `Retry microphone. ${status?.microphone.error ?? ""}`
- : "Turn microphone on"
- }
- aria-pressed={microphoneOn}
- title={microphoneOn ? "Microphone on" : "Microphone off"}
- onClick={() => void toggleMicrophone()}
+ <div
+ ref={microphoneControlRef}
+ className={`recording-microphone-split ${
+ microphoneOn ? "on" : ""
+ } ${microphoneError ? "error" : ""}`}
>
- <MicrophoneIcon off={!microphoneOn} />
- <span>{microphoneLabel}</span>
- </button>
+ <button
+ className="recording-microphone"
+ disabled={lifecycleBusy || microphoneBusy}
+ aria-label={
+ microphoneOn
+ ? `Mute ${activeMicrophoneLabel}`
+ : microphoneError
+ ? `Retry microphone. ${status?.microphone.error ?? ""}`
+ : `Unmute ${microphoneSettings?.selectedDeviceLabel ?? "System default"}`
+ }
+ aria-pressed={microphoneOn}
+ title={
+ microphoneOn
+ ? `Mute ${activeMicrophoneLabel}`
+ : `Unmute ${microphoneSettings?.selectedDeviceLabel ?? "System default"}`
+ }
+ onClick={() => void toggleMicrophone()}
+ >
+ <MicrophoneIcon off={!microphoneOn} />
+ <span>{microphoneLabel}</span>
+ </button>
+ <button
+ className="recording-microphone-menu-toggle"
+ disabled={lifecycleBusy || microphoneBusy}
+ aria-label="Choose microphone"
+ aria-haspopup="dialog"
+ aria-expanded={showMicrophoneMenu}
+ title="Choose microphone"
+ onClick={() => {
+ setActionError(null);
+ setConfirmDiscard(false);
+ setShowMicrophoneMenu((open) => !open);
+ }}
+ >
+ <ChevronIcon open={showMicrophoneMenu} />
+ </button>
+ </div>
<button
className="recording-discard"
disabled={lifecycleBusy}
onClick={() => {
setActionError(null);
+ setShowMicrophoneMenu(false);
setConfirmDiscard(true);
}}
>
@@ -208,6 +383,26 @@ export function RecordingControls() {
);
}
+function ChevronIcon({ open }: { open: boolean }) {
+ return (
+ <svg
+ width="12"
+ height="12"
+ viewBox="0 0 12 12"
+ fill="none"
+ aria-hidden
+ >
+ <path
+ d={open ? "m3 7.5 3-3 3 3" : "m3 4.5 3 3 3-3"}
+ stroke="currentColor"
+ strokeWidth="1.4"
+ strokeLinecap="round"
+ strokeLinejoin="round"
+ />
+ </svg>
+ );
+}
+
function MicrophoneIcon({ off }: { off: boolean }) {
return (
<svg width="16" height="16" viewBox="0 0 20 20" fill="none" aria-hidden>
diff --git a/src/WhatsRecorded.tsx b/src/WhatsRecorded.tsx
index 3f2880f..60b1790 100644
--- a/src/WhatsRecorded.tsx
+++ b/src/WhatsRecorded.tsx
@@ -72,19 +72,25 @@ export function WhatsRecorded({ onClose }: { onClose: () => void }) {
<section className="sheet-block">
<h3>Voice narration (only if you turn it on)</h3>
<p className="sheet-note">
- Off by default. Choose the initial state with Narrate, then use the floating recording bar.
+ Off by default. Choose the initial state and microphone with Narrate, then use the
+ floating recording bar.
</p>
<ul>
- <li>Your microphone is used only while it is on and a recording is running.</li>
+ <li>
+ Enabling Narrate briefly opens and releases the microphone so the app can request
+ permission and show the available inputs. No audio is saved during this check.
+ </li>
+ <li>Your microphone audio is saved only while it is on and a recording is running.</li>
<li>
Turning the microphone off ends that voice segment and releases the device. Turning it
- on again starts a new segment linked to the same recording timeline.
+ on again, or switching inputs, starts a new segment linked to the same recording
+ timeline.
</li>
<li>
The recording can be turned into text on this computer using an offline model. The
first transcription needs a one-time ~250 MB download that you choose when to start.
</li>
- <li>Leave the microphone off and it is never touched.</li>
+ <li>Leave Narrate and the recording-bar microphone off and no microphone is opened.</li>
</ul>
</section>