Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
feat(transcription): show each source's level and flag a silent source
Elian Dorand62e99cd6949
diff --git a/.claude/skills/working-with-transcription/SKILL.md b/.claude/skills/working-with-transcription/SKILL.mdindex e9e1016dff..87c9c4af0a 100644--- a/.claude/skills/working-with-transcription/SKILL.md+++ b/.claude/skills/working-with-transcription/SKILL.md@@ -22,14 +22,16 @@ capture window (hidden, offscreen) transcription process (Electron│ speakers: EmbeddingSpeakerTracker (TitaNet, synchronous),│ splits an utterance at a change of speakersinks: posted to the main process, dev: TranscriptFileSink- │ parentPort: ready / pending / utterance / log / stopped+ │ parentPort: ready / pending / level / utterance / log / stoppedmain process: TranscriptionWorker → AttachmentSink (transcript attachment)- state broadcast "transcription-state" (status, noteId, pendingSources, error)+ state broadcast "transcription-state" (status, noteId, pendingSources,+ stalledSources, error); each chunk's RMS on "transcription-level"```The main process opens the capture window, starts the transcription process and connects the two with a `MessageChannelMain`; per chunk it does nothing, per utterance it writes the attachment. Each recording gets a process of its own, which loads the model (~1.3 s) and exits after `stop`.- **Stopping** runs in order: `CaptureWindow.stop()` calls the page's `finishCapture()`, which flushes each worklet's partial chunk (padded with silence to whole 512-sample VAD windows) and posts `{ end: true }`; then the process, told to stop, waits for that marker (it travels on another port than the stop request, up to 2 s) before flushing the VADs and the decode queue.+- **A stalled source**: the main process marks a source in `stalledSources` once it has delivered no chunk for 10 s (`NO_AUDIO_MS`, checked every second) and clears it on the next chunk; a muted source still delivers silence. `TranscriptView` marks that source's level meter (fed by `onLevel()`, shown above the turns while its meeting records) _No audio_; the launcher turns its time badge red and shows an error once.- **Quitting** during a recording is held back (`before-quit`) until that stop finishes, at most 15 s.- **Failures** reach every window as a state with an `error`, which the launcher shows as a toast; `getState()` never returns it. The process dying ends the recording, keeping what was written; the capture page crashing or every source ending (`render-process-gone`, a track's `ended`) stops it normally, transcribing what was captured; a transcript that cannot be saved (a protected note while the protected session is locked) is reported once and kept in memory, and `AttachmentSink.save()` writes it on the next utterance or on `ENTER_PROTECTED_SESSION`.@@ -50,7 +52,7 @@ Nothing the transcription process imports may reach `@triliumnext/core` or `@tri| `packages/commons/src/lib/transcript.ts` | The `Transcript` format (`version: 1`, `startedAt`, `speakers`, `utterances`), `parseTranscript()`, speaker keys and `setSpeakerName()`; `cleanUtteranceText()`, the filler and stutter cleanup || Cleanup | The stored transcript stays verbatim. `TranscriptView` shows each utterance through `cleanUtteranceText()` while the synced option `transcriptCleanupEnabled` (default on, the "Hide filler words" switch above the transcript) holds, and leaves out utterances it empties. The rules are English-only for stutters (a function-word list) and keep numbers, comma-separated repeats and "that that"/"it it" || Speaker names | While a recording writes the transcript, `TranscriptView` renames through `electronApi.transcription.renameSpeaker(attachmentId, …)` (by attachment, since a note can hold an older transcript beside the running one): `AttachmentSink`, the only writer then, applies `setSpeakerName()` to the stored names in one transaction, so no utterance is overwritten. Otherwise it saves the stored transcript whole through the generic `POST notes/:noteId/attachments?matchBy=attachmentId`; attachments get no route of their own. `AttachmentSink` re-reads `speakers` before each rewrite, so a rename from another window or device survives the next utterance |-| `apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx` | Turns, clock times, the "Speaking…" line, fade-in |+| `apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx` | Turns, clock times, the "Speaking…" line, fade-in, the level meters || `apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx` | The launcher and `useTranscriptionState()` |## The modeldiff --git a/apps/client/src/translations/en/translation.json b/apps/client/src/translations/en/translation.jsonindex 08fb5cb97c..f1126e066c 100644--- a/apps/client/src/translations/en/translation.json+++ b/apps/client/src/translations/en/translation.json@@ -4604,7 +4604,10 @@"starting": "Starting the transcription…","stop": "Stop transcribing","stopping": "Finishing the transcript…",- "title_keeping_recordings": "{{title}} (keeps the audio for troubleshooting)"+ "title_keeping_recordings": "{{title}} (keeps the audio for troubleshooting)",+ "stop_no_audio": "Stop transcribing (no audio is arriving)",+ "no_audio_microphone": "No audio has arrived from your microphone for 10 seconds, so what you say is not being transcribed. Stop the transcription and start it again.",+ "no_audio_system": "No audio has arrived from the meeting for 10 seconds, so what the others say is not being transcribed. Stop the transcription and start it again."},"transcript": {"speaker_me": "Me",@@ -4616,7 +4619,8 @@"speaking": "Speaking…","offset": "{{offset}} into the transcription","cleanup": "Hide filler words",- "cleanup_tooltip": "Hides “uh”, “um” and stuttered words. The transcript itself is not changed."+ "cleanup_tooltip": "Hides “uh”, “um” and stuttered words. The transcript itself is not changed.",+ "no_audio": "No audio"},"resource_download": {"title": "Download required",diff --git a/apps/client/src/widgets/launch_bar/RecordMeetingButton.css b/apps/client/src/widgets/launch_bar/RecordMeetingButton.cssindex 3a149ac198..65b6bae7cc 100644--- a/apps/client/src/widgets/launch_bar/RecordMeetingButton.css+++ b/apps/client/src/widgets/launch_bar/RecordMeetingButton.css@@ -31,5 +31,10 @@color: var(--left-pane-item-selected-color, var(--main-text-color));box-shadow: var(--left-pane-item-selected-shadow);font-variant-numeric: tabular-nums;++ &.stalled {+ --color: var(--danger-color, #dc3545);+ color: #fff;+ }}}diff --git a/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsx b/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsxindex f142f14c48..f1342a50f4 100644--- a/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsx+++ b/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsx@@ -294,6 +294,50 @@ describe("RecordMeetingButton", () => {window.electronApi = undefined;});+ it("says once for each source that stops delivering audio, and marks the time", async () => {+ let onStateChanged: (state: TranscriptionState) => void = () => {};+ const recording: TranscriptionState = {+ status: "recording", noteId: "meeting", pendingSources: [], startedAt: Date.now()+ };+ const report = (state: Partial<TranscriptionState>) =>+ act(() => onStateChanged({ ...recording, ...state }));+ window.electronApi = {+ transcription: {+ isSupported: async () => true,+ getState: async () => recording,+ onStateChanged: (callback: typeof onStateChanged) => {+ onStateChanged = callback;+ return () => {};+ }+ }+ } as unknown as typeof window.electronApi;+ let container: HTMLElement | null = null;+ await act(async () => {+ container = renderInto(<RecordMeetingButton launcherNote={{} as FNote} />);+ });+ const badge = () => (container as HTMLElement | null)+ ?.querySelector(".record-meeting-time-badge");+ const button = () => (container as HTMLElement | null)?.querySelector("button");++ await vi.waitFor(() => expect(badge()).not.toBeNull());+ expect(badge()?.classList.contains("stalled")).toBe(false);+ await report({ stalledSources: [ "system" ] });+ await report({ stalledSources: [ "system" ] });+ expect(badge()?.classList.contains("stalled")).toBe(true);+ expect(button()?.textContent).toContain("record_meeting.stop_no_audio");+ expect(mocks.showError).toHaveBeenCalledExactlyOnceWith(+ expect.stringContaining("record_meeting.no_audio_system"));++ await report({});+ expect(badge()?.classList.contains("stalled")).toBe(false);+ expect(button()?.textContent).not.toContain("no_audio");+ await report({ stalledSources: [ "microphone", "system" ] });+ expect(mocks.showError).toHaveBeenCalledTimes(3);+ expect(mocks.showError).toHaveBeenCalledWith(+ expect.stringContaining("record_meeting.no_audio_microphone"));+ window.electronApi = undefined;+ });+it("shows the shared recording's state, and starts or stops it on a click", async () => {let onStateChanged: (state: TranscriptionState) => void = () => {};const report = (status: TranscriptionState["status"]) =>diff --git a/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx b/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsxindex 50beab05a6..59563bbb56 100644--- a/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx+++ b/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx@@ -2,9 +2,10 @@ import "./RecordMeetingButton.css";import {dayjs, getTranscriptionResources, MEETING_RECORDING_LABEL, type TranscriptionState,- type TranscriptionStatus+ type TranscriptionStatus, type TranscriptSource} from "@triliumnext/commons";-import { useCallback, useEffect, useState } from "preact/hooks";+import clsx from "clsx";+import { useCallback, useEffect, useRef, useState } from "preact/hooks";import dateNoteService from "../../services/date_notes";import { requireResources } from "../../services/dialog";@@ -24,11 +25,12 @@ import {* Launcher button that records a meeting: the first click creates the meeting note and starts the* recording, the second stops it. Desktop only, since the main process does the recording, and* shown only while `transcriptionEnabled` is on and the computer can transcribe. Its corner badge reads "Beta" while idle and the- * time elapsed while recording.+ * time elapsed while recording, which turns red, with a message once, when a source stops+ * delivering audio.*/export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps) {const { icon, title } = useLauncherIconAndTitle(launcherNote);- const { status, error, startedAt } = useTranscriptionState();+ const { status, error, startedAt, stalledSources } = useTranscriptionState();const [ isEnabled ] = useTriliumOptionBool("transcriptionEnabled");const isSupported = useTranscriptionSupported(isEnabled);const [ keepsRecordings ] = useTriliumOptionBool("transcriptionKeepRecordings");@@ -43,6 +45,8 @@ export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps)}}, [ error ]);+ useStalledSourceWarnings(stalledSources);+useEffect(() => {if (status !== "recording") {return;@@ -62,7 +66,7 @@ export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps)<LaunchBarActionButtonlauncherNote={launcherNote}onClick={handleClick}- {...getAppearance(status, icon, title, keepsRecordings)}+ {...getAppearance(status, icon, title, keepsRecordings, !!stalledSources?.length)}/>{status === "idle" && (<Badge text={t("common.beta")} className="record-meeting-beta-badge" />@@ -70,13 +74,31 @@ export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps){status === "recording" && startedAt !== undefined && (<Badgetext={formatElapsed(now - startedAt)}- className="record-meeting-time-badge"+ className={clsx("record-meeting-time-badge", {+ stalled: !!stalledSources?.length+ })}/>)}</div>);}+/** Shows an error once for each source that joins `stalledSources`. */+function useStalledSourceWarnings(stalledSources: TranscriptSource[] | undefined) {+ const warned = useRef<TranscriptSource[]>([]);+ const key = stalledSources?.join() ?? "";++ useEffect(() => {+ const stalled = stalledSources ?? [];+ for (const source of stalled.filter((source) => !warned.current.includes(source))) {+ toast.showError(source === "microphone"+ ? t("record_meeting.no_audio_microphone")+ : t("record_meeting.no_audio_system"));+ }+ warned.current = stalled;+ }, [ key ]);+}+const BUSY_ICON = "bx bx-loader-alt bx-spin";/**@@ -84,7 +106,8 @@ const BUSY_ICON = "bx bx-loader-alt bx-spin";* starting is when the people in the meeting would need to agree to it.*/function getAppearance(- status: TranscriptionStatus, idleIcon: string, idleTitle: string, keepsRecordings: boolean+ status: TranscriptionStatus, idleIcon: string, idleTitle: string, keepsRecordings: boolean,+ isStalled: boolean) {switch (status) {case "idle":@@ -99,7 +122,7 @@ function getAppearance(case "recording":return {icon: "bx bx-stop-circle",- text: t("record_meeting.stop"),+ text: isStalled ? t("record_meeting.stop_no_audio") : t("record_meeting.stop"),className: "record-meeting-active"};case "stopping":diff --git a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.css b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.cssindex 4f83e8af6c..99411610be 100644--- a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.css+++ b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.css@@ -14,6 +14,52 @@font-size: 0.9em;}+ .transcript-levels {+ display: flex;+ flex-wrap: wrap;+ align-items: center;+ gap: 0.25rem 1rem;+ margin-inline-end: auto;+ user-select: none;+ }++ .transcript-level {+ display: flex;+ align-items: center;+ gap: 0.35rem;++ .transcript-level-bar {+ position: relative;+ width: 4rem;+ height: 0.35rem;+ border-radius: 0.2rem;+ background: color-mix(in srgb, var(--main-text-color) 15%, transparent);+ overflow: hidden;++ &::after {+ content: "";+ position: absolute;+ inset-block: 0;+ inset-inline-start: 0;+ width: calc(var(--level, 0) * 100%);+ background: var(--main-text-color);+ transition: width 120ms linear;+ }+ }++ &.stalled {+ color: var(--danger-color, #dc3545);++ .transcript-level-bar {+ background: var(--danger-color, #dc3545);++ &::after {+ display: none;+ }+ }+ }+ }+.transcript-turn {max-width: var(--preferred-max-content-width, 900px);margin-bottom: 1rem;@@ -112,4 +158,8 @@.transcript .transcript-new {animation: none;}++ .transcript .transcript-level .transcript-level-bar::after {+ transition: none;+ }}diff --git a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsx b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsxindex cfc7d147c1..504a38b83f 100644--- a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsx+++ b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsx@@ -12,7 +12,7 @@ import options from "../../../services/options";import { renderInto } from "../../../test/render";import { ParentComponent } from "../../react/react_utils";import TranscriptView, {- formatOffset, groupIntoTurns, type TranscriptTurn+ formatOffset, groupIntoTurns, levelToMeter, type TranscriptTurn} from "./TranscriptView";const h = vi.hoisted(() => ({ post: vi.fn(), put: vi.fn() }));@@ -113,6 +113,14 @@ describe("transcript helpers", () => {]);});+ it("spreads 60 dB below full scale up to 10 dB below it over a level meter", () => {+ expect(levelToMeter(0)).toBe(0);+ expect(levelToMeter(10 ** (-60 / 20))).toBeCloseTo(0);+ expect(levelToMeter(10 ** (-35 / 20))).toBeCloseTo(0.5);+ expect(levelToMeter(10 ** (-10 / 20))).toBeCloseTo(1);+ expect(levelToMeter(1)).toBe(1);+ });+it("formats offsets as m:ss, with hours once there are any", () => {expect(formatOffset(0)).toBe("0:00");expect(formatOffset(65_900)).toBe("1:05");@@ -305,6 +313,7 @@ describe("TranscriptView", () => {transcription: {getState: async () => ({ status: "recording", noteId: "meeting", pendingSources: [] }),onStateChanged: () => () => {},+ onLevel: () => () => {},renameSpeaker}} as unknown as typeof window.electronApi;@@ -427,7 +436,8 @@ describe("TranscriptView", () => {onStateChanged: (callback: typeof onStateChanged) => {onStateChanged = callback;return () => {};- }+ },+ onLevel: () => () => {}}} as unknown as typeof window.electronApi;const getBlob = vi.fn()@@ -465,6 +475,64 @@ describe("TranscriptView", () => {window.electronApi = undefined;});+ it("shows each source's level while its meeting records, and which is silent", async () => {+ let onStateChanged: (state: TranscriptionState) => void = () => {};+ let onLevel: (source: TranscriptSource, level: number) => void = () => {};+ const unsubscribeLevels = vi.fn();+ const recording: TranscriptionState = {+ status: "recording", noteId: "meeting", pendingSources: []+ };+ const report = (state: Partial<TranscriptionState>) =>+ act(() => onStateChanged({ ...recording, ...state }));+ window.electronApi = {+ transcription: {+ getState: async () => recording,+ onStateChanged: (callback: typeof onStateChanged) => {+ onStateChanged = callback;+ return () => {};+ },+ onLevel: (callback: typeof onLevel) => {+ onLevel = callback;+ return unsubscribeLevels;+ }+ }+ } as unknown as typeof window.electronApi;+ const attachment = {+ attachmentId: "transcript1", ownerId: "meeting",+ getBlob: async () => ({ content: transcript([]) })+ };+ const container = renderInto(+ <ParentComponent.Provider value={new Component()}>+ <TranscriptView attachment={attachment as unknown as FAttachment} />+ </ParentComponent.Provider>+ );+ const meter = (source: string) => container.querySelector<HTMLElement>(+ `.transcript-level[data-source="${source}"]`);+ const fill = (source: string) =>+ Number(meter(source)?.style.getPropertyValue("--level"));++ await vi.waitFor(() => expect(meter("microphone")).not.toBeNull());+ expect(container.textContent).toContain("transcript.empty");+ expect(meter("microphone")?.textContent).toContain("transcript.speaker_me");+ expect(meter("system")?.textContent).toContain("transcript.speaker_others");+ expect(container.querySelector(".transcript-options input")).toBeNull();+ act(() => onLevel("microphone", 10 ** (-35 / 20)));+ expect(fill("microphone")).toBeCloseTo(0.5);+ expect(fill("system")).toBe(0);+ expect(meter("microphone")?.getAttribute("aria-valuenow")).toBe("0.5");++ await report({ stalledSources: [ "system" ] });+ expect(meter("system")?.classList.contains("stalled")).toBe(true);+ expect(meter("system")?.textContent).toContain("transcript.no_audio");+ expect(meter("microphone")?.classList.contains("stalled")).toBe(false);+ expect(meter("microphone")?.textContent).not.toContain("transcript.no_audio");++ await report({ noteId: "other" });+ expect(meter("microphone")).toBeNull();+ expect(unsubscribeLevels).toHaveBeenCalledOnce();+ window.electronApi = undefined;+ });+it("follows new turns for a reader at the end, not for one who scrolled up", async () => {const utterance = (index: number): TranscriptUtterance =>({ source: "microphone", offsetMs: index * 60_000, text: `Line ${index}` });diff --git a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsxindex 9ca887c5f1..1eeea630ae 100644--- a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx+++ b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx@@ -4,6 +4,7 @@ import {cleanUtteranceText, getSpeakerKey, MAX_SPEAKER_NAME_LENGTH, parseTranscript, setSpeakerName,type TranscriptSource, type TranscriptUtterance} from "@triliumnext/commons";+import clsx from "clsx";import { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from "preact/hooks";import type FAttachment from "../../../entities/fattachment";@@ -31,14 +32,17 @@ const PARAGRAPH_PAUSE_MS = 2000;* Shows a `transcript` attachment as the conversation it records, one paragraph per speaker turn.* Follows the attachment as a running recording rewrites it, shows which speakers the recording* hears before their words are transcribed, and keeps a reader who is at the end there as new- * turns arrive. With `transcriptCleanupEnabled`, shows each utterance through+ * turns arrive. While its meeting records, shows each source's level above the turns, and which+ * source delivers no audio. With `transcriptCleanupEnabled`, shows each utterance through* `cleanUtteranceText()` and leaves out the ones it empties; the stored transcript stays verbatim.*/export default function TranscriptView({ attachment }: { attachment: FAttachment }) {const [ content, setContent ] = useState<string | null>();const [ isCleanedUp, setCleanedUp ] = useTriliumOptionBool("transcriptCleanupEnabled");const recording = useTranscriptionState();- const pending = recording.noteId === attachment.ownerId ? recording.pendingSources : [];+ const isOwnRecording = recording.noteId === attachment.ownerId;+ const pending = isOwnRecording ? recording.pendingSources : [];+ const isRecording = isOwnRecording && recording.status === "recording";const scrollerRef = useRef<HTMLDivElement>(null);/** The scroll height at the previous render, or `null` before the transcript first shows. */const scrollHeightRef = useRef<number | null>(null);@@ -86,8 +90,25 @@ export default function TranscriptView({ attachment }: { attachment: FAttachmentif (!transcript) {return <NoItems icon="bx bx-error" text={t("transcript.unreadable")} />;}- if (!transcript.utterances.length && !pending.length) {- return <NoItems icon="bx bx-microphone" text={t("transcript.empty")} />;+ const hasUtterances = transcript.utterances.length > 0;+ const options = (hasUtterances || isRecording) && (+ <div className="transcript-options">+ {isRecording && <TranscriptLevels stalledSources={recording.stalledSources ?? []} />}+ {hasUtterances && (+ <FormToggle+ switchOnName={t("transcript.cleanup")}+ switchOffName={t("transcript.cleanup")}+ switchOnTooltip={t("transcript.cleanup_tooltip")}+ switchOffTooltip={t("transcript.cleanup_tooltip")}+ currentValue={isCleanedUp}+ onChange={setCleanedUp}+ />+ )}+ </div>+ );+ if (!hasUtterances && !pending.length) {+ const empty = <NoItems icon="bx bx-microphone" text={t("transcript.empty")} />;+ return options ? <div className="transcript">{options}{empty}</div> : empty;}initialCountRef.current ??= transcript.utterances.length;@@ -110,18 +131,7 @@ export default function TranscriptView({ attachment }: { attachment: FAttachment);return (<div ref={scrollerRef} className="transcript">- {transcript.utterances.length > 0 && (- <div className="transcript-options">- <FormToggle- switchOnName={t("transcript.cleanup")}- switchOffName={t("transcript.cleanup")}- switchOnTooltip={t("transcript.cleanup_tooltip")}- switchOffTooltip={t("transcript.cleanup_tooltip")}- currentValue={isCleanedUp}- onChange={setCleanedUp}- />- </div>- )}+ {options}{turns.map((turn) => (<div key={turn.offsetMs} className={`transcript-turn transcript-${turn.source}`}><div className="transcript-turn-header">@@ -165,6 +175,62 @@ export default function TranscriptView({ attachment }: { attachment: FAttachment);}+const LEVEL_ICONS: Record<TranscriptSource, string> = {+ microphone: "bx-microphone",+ system: "bx-volume-full"+};+const LEVEL_SOURCES = Object.keys(LEVEL_ICONS) as TranscriptSource[];++/**+ * One meter per source, named as the transcript names it, filled to its latest level, and saying+ * so when the source delivers no audio.+ */+function TranscriptLevels({ stalledSources }: { stalledSources: TranscriptSource[] }) {+ const levels = useTranscriptionLevels();+ return (+ <div className="transcript-levels">+ {LEVEL_SOURCES.map((source) => {+ const level = levelToMeter(levels[source]);+ const isStalled = stalledSources.includes(source);+ return (+ <span+ key={source}+ data-source={source}+ className={clsx("transcript-level", { stalled: isStalled })}+ role="meter"+ aria-label={speakerName(source)}+ aria-valuemin={0}+ aria-valuemax={1}+ aria-valuenow={Math.round(level * 100) / 100}+ style={{ "--level": level }}+ >+ <span className={`bx ${LEVEL_ICONS[source]}`} aria-hidden="true" />+ {speakerName(source)}+ <span className="transcript-level-bar" aria-hidden="true" />+ {isStalled && (+ <span className="transcript-level-warning">+ {t("transcript.no_audio")}+ </span>+ )}+ </span>+ );+ })}+ </div>+ );+}++/** Each source's latest level, from `electronApi.transcription.onLevel()`. */+function useTranscriptionLevels() {+ const [ levels, setLevels ] = useState<Record<TranscriptSource, number>>({+ microphone: 0, system: 0+ });++ useEffect(() => window.electronApi?.transcription.onLevel((source, level) =>+ setLevels((current) => ({ ...current, [source]: level }))), []);++ return levels;+}+/*** A speaker's name, which a click turns into a field that renames every turn of the speaker: Enter* or leaving the field saves, Escape cancels, and a blank name or the default restores the default.@@ -324,6 +390,23 @@ export function groupIntoTurns(}/** `m:ss`, or `h:mm:ss` from the first hour on. */+/** The quietest and the loudest RMS a level meter shows, in dB below full scale. */+const METER_FLOOR_DB = -60;+const METER_CEILING_DB = -10;++/**+ * Maps an RMS from 0 to 1 onto a level meter's 0 to 1, in decibels: speech, about 35 dB below full+ * scale, fills half of it.+ */+export function levelToMeter(level: number) {+ if (level <= 0) {+ return 0;+ }+ const db = 20 * Math.log10(level);+ const fraction = (db - METER_FLOOR_DB) / (METER_CEILING_DB - METER_FLOOR_DB);+ return Math.min(1, Math.max(0, fraction));+}+export function formatOffset(milliseconds: number) {const totalSeconds = Math.max(0, Math.floor(milliseconds / 1000));const hours = Math.floor(totalSeconds / 3600);diff --git a/apps/desktop/src/preload.spec.ts b/apps/desktop/src/preload.spec.tsindex 19ea491a2b..1f2ab389f8 100644--- a/apps/desktop/src/preload.spec.ts+++ b/apps/desktop/src/preload.spec.ts@@ -756,5 +756,18 @@ describe("preload script", () => {unsubscribe();expect(ipcRendererListeners.has("transcription-state")).toBe(false);});++ it("onLevel relays each source's level until unsubscribed", () => {+ const callback = vi.fn();+ const unsubscribe = transcription().onLevel(callback);++ const listeners = ipcRendererListeners.get("transcription-level") ?? [];+ expect(listeners).toHaveLength(1);+ listeners[0]({}, "microphone", 0.25);+ expect(callback).toHaveBeenCalledExactlyOnceWith("microphone", 0.25);++ unsubscribe();+ expect(ipcRendererListeners.has("transcription-level")).toBe(false);+ });});});diff --git a/apps/desktop/src/preload.ts b/apps/desktop/src/preload.tsindex fdd2f49d6d..387f931101 100644--- a/apps/desktop/src/preload.ts+++ b/apps/desktop/src/preload.ts@@ -6,7 +6,8 @@ import type {NativeImportOptions,OneNoteLoginResult,RendererStartupMetric,- TranscriptionState+ TranscriptionState,+ TranscriptSource} from "@triliumnext/commons";import { contextBridge, ipcRenderer, webFrame, webUtils } from "electron";@@ -277,6 +278,12 @@ contextBridge.exposeInMainWorld("electronApi", {const listener = (_event: unknown, state: TranscriptionState) => callback(state);ipcRenderer.on("transcription-state", listener);return () => ipcRenderer.removeListener("transcription-state", listener);+ },+ onLevel(callback: (source: TranscriptSource, level: number) => void) {+ const listener = (_event: unknown, source: TranscriptSource, level: number) =>+ callback(source, level);+ ipcRenderer.on("transcription-level", listener);+ return () => ipcRenderer.removeListener("transcription-level", listener);}},diff --git a/apps/desktop/src/services/transcription/index.spec.ts b/apps/desktop/src/services/transcription/index.spec.tsindex dd1b06fc5f..d9d353a374 100644--- a/apps/desktop/src/services/transcription/index.spec.ts+++ b/apps/desktop/src/services/transcription/index.spec.ts@@ -1,6 +1,6 @@import type { TranscriptionState } from "@triliumnext/commons";import path from "path";-import { beforeEach, describe, expect, it, vi } from "vitest";+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";import type { CaptureWindowOptions } from "./capture";import type { WorkerListeners } from "./worker_client";@@ -10,6 +10,7 @@ const h = vi.hoisted(() => ({appListeners: new Map<string, (...args: unknown[]) => unknown>(),quit: vi.fn(),broadcast: [] as TranscriptionState[],+ levels: [] as [ string, number ][],notes: new Set<string>([ "meeting" ]),isProtected: false,isProtectedSessionAvailable: false,@@ -42,9 +43,11 @@ vi.mock("electron", () => ({BrowserWindow: {getAllWindows: () => [ 1, 2 ].map(() => ({webContents: {- send: (channel: string, state: TranscriptionState) => {+ send: (channel: string, ...args: unknown[]) => {if (channel === "transcription-state") {- h.broadcast.push(state);+ h.broadcast.push(args[0] as TranscriptionState);+ } else if (channel === "transcription-level") {+ h.levels.push(args as [ string, number ]);}}}@@ -142,6 +145,7 @@ beforeEach(() => {h.isProtected = false;h.isProtectedSessionAvailable = false;h.broadcast.length = 0;+ h.levels.length = 0;h.keepRecordings = false;h.transcriptionModel = null;h.speakersEnabled = true;@@ -155,6 +159,10 @@ beforeEach(() => {h.capture.stop.mockResolvedValue(undefined);});+afterEach(() => {+ vi.useRealTimers();+});+describe("transcription IPC", () => {it("records one meeting at a time and tells every window about each step", async () => {const invoke = await loadService();@@ -239,6 +247,49 @@ describe("transcription IPC", () => {expect(h.broadcast.at(-1)).toEqual(IDLE);});+ it("passes each source's level on to every window", async () => {+ const invoke = await loadService();+ await invoke("transcription-start", "meeting");+ h.listeners?.onLevel("system", 0.125);+ expect(h.levels).toEqual([ [ "system", 0.125 ], [ "system", 0.125 ] ]);+ });++ it("flags a source no audio has come from for 10 s, until it comes again", async () => {+ const invoke = await loadService();+ vi.useFakeTimers();+ await invoke("transcription-start", "meeting");+ const recording = {+ status: "recording", noteId: "meeting", pendingSources: [], startedAt: STARTED_AT+ };+ const tick = async () => {+ await vi.advanceTimersByTimeAsync(1000);+ h.listeners?.onLevel("system", 0.1);+ };++ for (let second = 1; second < 10; second++) {+ await tick();+ }+ expect(h.broadcast.at(-1)).toEqual(recording);+ await tick();+ expect(h.broadcast.at(-1)).toEqual({ ...recording, stalledSources: [ "microphone" ] });+ expect(await invoke("transcription-get-state")).toEqual(h.broadcast.at(-1));+ expect(h.log.error).toHaveBeenCalledExactlyOnceWith(+ "Transcription: no audio from the microphone for 10 s.");+ const flagged = h.broadcast.length;+ await tick();+ expect(h.broadcast).toHaveLength(flagged);++ h.listeners?.onLevel("microphone", 0);+ expect(h.broadcast.at(-1)).toEqual(recording);+ expect(h.log.info).toHaveBeenCalledWith("Transcription: audio from the microphone again.");++ await invoke("transcription-stop");+ const stopped = h.broadcast.length;+ await vi.advanceTimersByTimeAsync(20_000);+ expect(h.broadcast).toHaveLength(stopped);+ vi.useRealTimers();+ });+it("says once that a locked protected session holds back the transcript, and saves it on unlocking", async () => {h.isProtected = true;const invoke = await loadService();diff --git a/apps/desktop/src/services/transcription/index.ts b/apps/desktop/src/services/transcription/index.tsindex 93d365ee64..0e4082db56 100644--- a/apps/desktop/src/services/transcription/index.ts+++ b/apps/desktop/src/services/transcription/index.ts@@ -1,4 +1,6 @@-import type { TranscriptionState, TranscriptUtterance } from "@triliumnext/commons";+import type {+ TranscriptionState, TranscriptSource, TranscriptUtterance+} from "@triliumnext/commons";import { becca, events, getLog, options, protected_session } from "@triliumnext/core";import dataDirs from "@triliumnext/server/src/services/data_dir.js";import resources from "@triliumnext/server/src/services/resources.js";@@ -23,9 +25,18 @@ const IDLE: TranscriptionState = { status: "idle", noteId: null, pendingSources:/** How long quitting waits for a running transcription to finish. */const QUIT_TIMEOUT_MS = 15_000;+/**+ * How long a source can go without delivering a chunk before it counts as stalled. A silent+ * source still delivers chunks of silence, about four a second.+ */+const NO_AUDIO_MS = 10_000;+interface Recording {worker: TranscriptionWorker;capture: CaptureWindow;+ /** When each source last delivered a chunk, in milliseconds since the epoch. */+ lastHeard: Map<TranscriptSource, number>;+ audioCheck?: ReturnType<typeof setInterval>;}let state = IDLE;@@ -87,9 +98,11 @@ export async function startRecording(noteId: unknown): Promise<string> {worker: new TranscriptionWorker({onUtterance: (utterance) => writeUtterance(sink, noteId, utterance),onPendingChanged: (pendingSources) => setState({ ...state, pendingSources }),+ onLevel: (source, level) => handleLevel(recording, source, level),onUnexpectedExit: (code) => handleUnexpectedExit(recording, code),log: getLog()- })+ }),+ lastHeard: new Map()};activeRecording = recording;try {@@ -106,11 +119,16 @@ export async function startRecording(noteId: unknown): Promise<string> {recording.worker.connectCapture(port2);const sources = await recording.capture.start(port1);getLog().info(`Transcription: recording ${sources.join(" and ")} for note ${noteId}.`);+ for (const source of sources) {+ recording.lastHeard.set(source, Date.now());+ }+ recording.audioCheck = setInterval(() => checkAudio(recording), 1000);setState({ ...state, status: "recording", startedAt });return "";} catch (e) {activeRecording = null;writer = null;+ clearInterval(recording.audioCheck);void recording.capture.stop();recording.worker.kill();setState(IDLE);@@ -131,6 +149,7 @@ export function stopRecording(error?: string): Promise<void> {setState({ ...state, status: "stopping" });activeRecording = null;+ clearInterval(recording.audioCheck);const { noteId } = state;stopping = recording.capture.stop().then(() => recording.worker.stop())@@ -273,12 +292,53 @@ function handleUnexpectedExit(recording: Recording, code: number) {}activeRecording = null;writer = null;+ clearInterval(recording.audioCheck);void recording.capture.stop();getLog().error(`Transcription: the transcription process exited with code ${code}.`);state = IDLE;broadcast({ ...IDLE, error: t("transcription.stopped-unexpectedly") });}+/** Passes a source's level on to every window, and clears that source's stall. */+function handleLevel(recording: Recording, source: TranscriptSource, level: number) {+ if (activeRecording !== recording) {+ return;+ }+ recording.lastHeard.set(source, Date.now());+ if (state.stalledSources?.includes(source)) {+ getLog().info(`Transcription: audio from the ${source} again.`);+ setStalledSources(state.stalledSources.filter((stalled) => stalled !== source));+ }+ for (const window of BrowserWindow.getAllWindows()) {+ window.webContents.send("transcription-level", source, level);+ }+}++/** Reports the sources that have delivered nothing for {@link NO_AUDIO_MS}. */+function checkAudio(recording: Recording) {+ if (activeRecording !== recording || state.status !== "recording") {+ return;+ }+ const now = Date.now();+ const stalled = [ ...recording.lastHeard ]+ .filter(([ , lastHeard ]) => now - lastHeard >= NO_AUDIO_MS)+ .map(([ source ]) => source)+ .sort();+ const current = state.stalledSources ?? [];+ if (stalled.join() === current.join()) {+ return;+ }+ for (const source of stalled.filter((source) => !current.includes(source))) {+ getLog().error(`Transcription: no audio from the ${source} for ${NO_AUDIO_MS / 1000} s.`);+ }+ setStalledSources(stalled);+}++function setStalledSources(stalledSources: TranscriptSource[]) {+ const { stalledSources: _, ...rest } = state;+ setState(stalledSources.length ? { ...rest, stalledSources } : rest);+}+function setState(newState: TranscriptionState) {state = newState;broadcast(state);diff --git a/apps/desktop/src/services/transcription/protocol.ts b/apps/desktop/src/services/transcription/protocol.tsindex 607b819761..a16dd4ff47 100644--- a/apps/desktop/src/services/transcription/protocol.ts+++ b/apps/desktop/src/services/transcription/protocol.ts@@ -41,6 +41,8 @@ export type WorkerEvent =| { type: "failed"; message: string }| { type: "pending"; sources: TranscriptSource[] }| { type: "utterance"; utterance: TranscriptUtterance }+ /** The RMS of a source's latest chunk, from 0 to 1; one per chunk, about four a second. */+ | { type: "level"; source: TranscriptSource; level: number }| { type: "log"; level: keyof Logger; message: string }/** Every utterance captured before `stop` has been sent. */| { type: "stopped" }diff --git a/apps/desktop/src/services/transcription/session.spec.ts b/apps/desktop/src/services/transcription/session.spec.tsindex 4f8efe7b99..6c7c9b7131 100644--- a/apps/desktop/src/services/transcription/session.spec.ts+++ b/apps/desktop/src/services/transcription/session.spec.ts@@ -80,12 +80,13 @@ function createHarness(extraSinks: TranscriptSink[] = [], taps: AudioTap[] = [])};const sinks = [ ...extraSinks, sink ];const onPendingChanged = vi.fn();+ const onLevel = vi.fn();const session = new RecordingSession({- capture, engine, sinks, taps, log, onPendingChanged, now: () => clock+ capture, engine, sinks, taps, log, onPendingChanged, onLevel, now: () => clock});return {- session, capture, engine, sink, written, flushed, onPendingChanged,+ session, capture, engine, sink, written, flushed, onPendingChanged, onLevel,/** Delivers `samples` from `source` when the clock reads `at`. */deliver(source: TranscriptSource, samples: Float32Array, at: number) {clock = at;@@ -276,6 +277,22 @@ describe("RecordingSession", () => {await session.stop();});+ it("reports the loudness of every chunk as it arrives, per source", async () => {+ const { session, deliver, onLevel } = createHarness();+ await session.start();+ const loud = speechChunk(true).fill(0.5, 3);+ deliver("microphone", speechChunk(false), 2000);+ deliver("system", loud, 3000);++ expect(onLevel).toHaveBeenCalledTimes(2);+ const [ [ quietSource, quiet ], [ loudSource, level ] ] = onLevel.mock.calls;+ expect(quietSource).toBe("microphone");+ expect(quiet).toBeCloseTo(Math.sqrt(1 / 16000));+ expect(loudSource).toBe("system");+ expect(level).toBeCloseTo(Math.sqrt((2 + 0.25 * 15997) / 16000));+ await session.stop();+ });+it("logs a failed transcription or tap, and still finishes the rest", async () => {const tap = { write: vi.fn(), close: vi.fn(() => {throw new Error("disk full");diff --git a/apps/desktop/src/services/transcription/session.ts b/apps/desktop/src/services/transcription/session.tsindex 22b8c6a3ca..ed5f4dbb5d 100644--- a/apps/desktop/src/services/transcription/session.ts+++ b/apps/desktop/src/services/transcription/session.ts@@ -53,6 +53,8 @@ export interface RecordingSessionOptions {* still being spoken or because it waits to be transcribed, every time that list changes.*/onPendingChanged?: (sources: TranscriptSource[]) => void;+ /** Called with the RMS of every chunk as it arrives, from 0 to 1. */+ onLevel?: (source: TranscriptSource, level: number) => void;now?: () => number;}@@ -87,7 +89,7 @@ export class RecordingSession {try {const pipeline = this.pipelineFor(source, samples);this.tap(source, samples, pipeline.firstSampleOffsetMs);- pipeline.accept(samples);+ this.options.onLevel?.(source, pipeline.accept(samples));this.updatePending();} catch (e) {this.log.error(`Transcription: ${source} failed: ${(e as Error)?.message ?? e}`);@@ -228,7 +230,10 @@ export class RecordingSession {}}-/** Segments one source and logs its peak level every 5 s, to spot a silent source. */+/**+ * Segments one source, measures each chunk's RMS and logs the source's peak level every 5 s, to+ * spot a silent source.+ */class SourcePipeline {private peak = 0;private peakSampleCount = 0;@@ -242,9 +247,11 @@ class SourcePipeline {private readonly log: Logger) {}+ /** Segments `samples` and returns their RMS. */accept(samples: Float32Array) {- this.trackPeak(samples);+ const level = this.measure(samples);this.emit(this.segmenter.accept(samples));+ return level;}flush() {@@ -262,9 +269,11 @@ class SourcePipeline {}}- private trackPeak(samples: Float32Array) {+ private measure(samples: Float32Array) {+ let sumOfSquares = 0;for (const sample of samples) {this.peak = Math.max(this.peak, Math.abs(sample));+ sumOfSquares += sample * sample;}this.peakSampleCount += samples.length;if (this.peakSampleCount >= 5 * SAMPLE_RATE) {@@ -272,6 +281,7 @@ class SourcePipeline {this.peak = 0;this.peakSampleCount = 0;}+ return samples.length ? Math.sqrt(sumOfSquares / samples.length) : 0;}}diff --git a/apps/desktop/src/services/transcription/transcription_worker.ts b/apps/desktop/src/services/transcription/transcription_worker.tsindex 3e3e3c32b9..03a0a7cdf3 100644--- a/apps/desktop/src/services/transcription/transcription_worker.ts+++ b/apps/desktop/src/services/transcription/transcription_worker.ts@@ -65,7 +65,8 @@ async function start({ noteId, models, recordingDirectory }: StartRequest) {sinks,taps,log,- onPendingChanged: (sources) => post({ type: "pending", sources })+ onPendingChanged: (sources) => post({ type: "pending", sources }),+ onLevel: (source, level) => post({ type: "level", source, level })});await newSession.start();session = newSession;diff --git a/apps/desktop/src/services/transcription/worker_client.spec.ts b/apps/desktop/src/services/transcription/worker_client.spec.tsindex ef44414fdd..d4913a9bcf 100644--- a/apps/desktop/src/services/transcription/worker_client.spec.ts+++ b/apps/desktop/src/services/transcription/worker_client.spec.ts@@ -44,6 +44,7 @@ beforeEach(() => {listeners = {onUtterance: vi.fn(),onPendingChanged: vi.fn(),+ onLevel: vi.fn(),onUnexpectedExit: vi.fn(),log: { info: vi.fn(), error: vi.fn() }};@@ -96,6 +97,8 @@ describe("TranscriptionWorker", () => {child.report({ type: "utterance", utterance });child.report({ type: "pending", sources: [ "system" ] });child.report({ type: "log", level: "error", message: "system failed" });+ child.report({ type: "level", source: "microphone", level: 0.25 });+ expect(listeners.onLevel).toHaveBeenCalledExactlyOnceWith("microphone", 0.25);expect(listeners.onUtterance).toHaveBeenCalledExactlyOnceWith(utterance);expect(listeners.onPendingChanged).toHaveBeenCalledExactlyOnceWith([ "system" ]);expect(listeners.log.error).toHaveBeenCalledExactlyOnceWith("system failed");diff --git a/apps/desktop/src/services/transcription/worker_client.ts b/apps/desktop/src/services/transcription/worker_client.tsindex ee237bc48b..b820b045ff 100644--- a/apps/desktop/src/services/transcription/worker_client.ts+++ b/apps/desktop/src/services/transcription/worker_client.ts@@ -8,6 +8,8 @@ import type { Logger, WorkerEvent, WorkerRequest } from "./protocol";export interface WorkerListeners {onUtterance(utterance: TranscriptUtterance): void;onPendingChanged(sources: TranscriptSource[]): void;+ /** Called with the RMS of each chunk a source delivers, from 0 to 1. */+ onLevel(source: TranscriptSource, level: number): void;/** Called when the process ends without having been stopped, as when it crashes. */onUnexpectedExit(code: number): void;log: Logger;@@ -94,6 +96,9 @@ export class TranscriptionWorker {case "utterance":this.listeners.onUtterance(event.utterance);break;+ case "level":+ this.listeners.onLevel(event.source, event.level);+ break;case "log":this.listeners.log[event.level](event.message);break;diff --git a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.html b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.htmlindex f86e19d675..3cb8fad848 100644--- a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.html+++ b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.html@@ -45,6 +45,11 @@<p>Only one meeting can be transcribed at a time. The note itself is an ordinarytext note: you can take notes in it during the meeting, rename it or moveit anywhere in the tree. It carries the <code>#meetingRecording</code> label.</p>+<p>If no audio arrives from your microphone or from the meeting for 10 seconds,+ Trilium shows a message, the time on the button turns red and the button's+ tooltip says that no audio is arriving. Silence does not cause this: a+ muted microphone still delivers audio. Stop the transcription and start+ it again; what was transcribed until then is kept.</p><p>If the transcription stops on its own, for example because the audio couldno longer be captured, Trilium shows a message and keeps everything transcribeduntil then. Closing Trilium during a meeting waits for the transcription@@ -53,9 +58,13 @@<p>The transcript is stored as an <a href="#root/_help_0vhv7lsOLy82">attachment</a> ofthe meeting note and is shown in a collapsible <em>Transcript</em> sectionabove the note's content. It updates as the meeting goes on, one paragraph- per speaker turn, each with the time it started. While someone is talking, <em>Speaking…</em> shows- under their name until their words are transcribed. If you are reading- the end of the transcript, it scrolls to follow new text.</p>+ per speaker turn, each with the time it started. While the meeting is being+ transcribed, a meter above the transcript shows the level of your microphone+ (<em>Me</em>) and of the meeting's audio (<em>Others</em>), so you can+ tell that both are being heard; a source that delivers no audio is marked <em>No audio</em>.+ While someone is talking, <em>Speaking…</em> shows under their name until+ their words are transcribed. If you are reading the end of the transcript,+ it scrolls to follow new text.</p><ul><li>What you say into your microphone is labeled <em>Me</em>.</li><li>What the other participants say is labeled <em>Speaker 1</em>, <em>Speaker 2</em> anddiff --git a/docs/User Guide/User Guide/Meeting transcription.md b/docs/User Guide/User Guide/Meeting transcription.mdindex 8fcd777745..f1263014cf 100644--- a/docs/User Guide/User Guide/Meeting transcription.md+++ b/docs/User Guide/User Guide/Meeting transcription.md@@ -25,11 +25,13 @@ The speech recognition models are not part of Trilium. The first time you transcOnly one meeting can be transcribed at a time. The note itself is an ordinary text note: you can take notes in it during the meeting, rename it or move it anywhere in the tree. It carries the `#meetingRecording` label.+If no audio arrives from your microphone or from the meeting for 10 seconds, Trilium shows a message, the time on the button turns red and the button's tooltip says that no audio is arriving. Silence does not cause this: a muted microphone still delivers audio. Stop the transcription and start it again; what was transcribed until then is kept.+If the transcription stops on its own, for example because the audio could no longer be captured, Trilium shows a message and keeps everything transcribed until then. Closing Trilium during a meeting waits for the transcription to finish first.## The transcript-The transcript is stored as an [attachment](Basic%20Concepts%20and%20Features/Notes/Attachments.md) of the meeting note and is shown in a collapsible _Transcript_ section above the note's content. It updates as the meeting goes on, one paragraph per speaker turn, each with the time it started. While someone is talking, _Speaking…_ shows under their name until their words are transcribed. If you are reading the end of the transcript, it scrolls to follow new text.+The transcript is stored as an [attachment](Basic%20Concepts%20and%20Features/Notes/Attachments.md) of the meeting note and is shown in a collapsible _Transcript_ section above the note's content. It updates as the meeting goes on, one paragraph per speaker turn, each with the time it started. While the meeting is being transcribed, a meter above the transcript shows the level of your microphone (_Me_) and of the meeting's audio (_Others_), so you can tell that both are being heard; a source that delivers no audio is marked _No audio_. While someone is talking, _Speaking…_ shows under their name until their words are transcribed. If you are reading the end of the transcript, it scrolls to follow new text.* What you say into your microphone is labeled _Me_.* What the other participants say is labeled _Speaker 1_, _Speaker 2_ and so on, while _Identify speakers_ is on; otherwise it is labeled _Others_.diff --git a/packages/commons/src/lib/electron_api_interface.ts b/packages/commons/src/lib/electron_api_interface.tsindex 34b0f5b70b..303402bb24 100644--- a/packages/commons/src/lib/electron_api_interface.ts+++ b/packages/commons/src/lib/electron_api_interface.ts@@ -724,6 +724,11 @@ export interface TranscriptionState {noteId: string | null;/** The sources heard speaking, or whose speech is still being transcribed. */pendingSources: TranscriptSource[];+ /**+ * The sources that have delivered no audio for 10 s while recording, as when their capture+ * hangs; absent when every source delivers. A muted source still delivers silence.+ */+ stalledSources?: TranscriptSource[];/*** When the recording started, in milliseconds since the epoch: set once it records, kept while* it stops, absent otherwise.@@ -780,6 +785,14 @@ export interface ElectronTranscriptionApi {* @returns A function that removes the listener.*/onStateChanged(callback: (state: TranscriptionState) => void): () => void;++ /**+ * Calls `callback` with the RMS, from 0 to 1, of every chunk a source of the running recording+ * delivers, about four times a second per source.+ *+ * @returns A function that stops the calls.+ */+ onLevel(callback: (source: TranscriptSource, level: number) => void): () => void;}/**