Skip to content

Build your personal knowledge base with Trilium Notes

feat(transcription): show each source's level and flag a silent source

Elian Dorand62e99cd6949
diff --git a/.claude/skills/working-with-transcription/SKILL.md b/.claude/skills/working-with-transcription/SKILL.md
index e9e1016dff..87c9c4af0a 100644
--- a/.claude/skills/working-with-transcription/SKILL.md
+++ b/.claude/skills/working-with-transcription/SKILL.md
@@ -22,14 +22,16 @@ capture window (hidden, offscreen) transcription process (Electron
│ speakers: EmbeddingSpeakerTracker (TitaNet, synchronous),
│ splits an utterance at a change of speaker
sinks: posted to the main process, dev: TranscriptFileSink
- │ parentPort: ready / pending / utterance / log / stopped
+ │ parentPort: ready / pending / level / utterance / log / stopped
main process: TranscriptionWorker → AttachmentSink (transcript attachment)
- state broadcast "transcription-state" (status, noteId, pendingSources, error)
+ state broadcast "transcription-state" (status, noteId, pendingSources,
+ stalledSources, error); each chunk's RMS on "transcription-level"
```
The main process opens the capture window, starts the transcription process and connects the two with a `MessageChannelMain`; per chunk it does nothing, per utterance it writes the attachment. Each recording gets a process of its own, which loads the model (~1.3 s) and exits after `stop`.
- **Stopping** runs in order: `CaptureWindow.stop()` calls the page's `finishCapture()`, which flushes each worklet's partial chunk (padded with silence to whole 512-sample VAD windows) and posts `{ end: true }`; then the process, told to stop, waits for that marker (it travels on another port than the stop request, up to 2 s) before flushing the VADs and the decode queue.
+- **A stalled source**: the main process marks a source in `stalledSources` once it has delivered no chunk for 10 s (`NO_AUDIO_MS`, checked every second) and clears it on the next chunk; a muted source still delivers silence. `TranscriptView` marks that source's level meter (fed by `onLevel()`, shown above the turns while its meeting records) _No audio_; the launcher turns its time badge red and shows an error once.
- **Quitting** during a recording is held back (`before-quit`) until that stop finishes, at most 15 s.
- **Failures** reach every window as a state with an `error`, which the launcher shows as a toast; `getState()` never returns it. The process dying ends the recording, keeping what was written; the capture page crashing or every source ending (`render-process-gone`, a track's `ended`) stops it normally, transcribing what was captured; a transcript that cannot be saved (a protected note while the protected session is locked) is reported once and kept in memory, and `AttachmentSink.save()` writes it on the next utterance or on `ENTER_PROTECTED_SESSION`.
@@ -50,7 +52,7 @@ Nothing the transcription process imports may reach `@triliumnext/core` or `@tri
| `packages/commons/src/lib/transcript.ts` | The `Transcript` format (`version: 1`, `startedAt`, `speakers`, `utterances`), `parseTranscript()`, speaker keys and `setSpeakerName()`; `cleanUtteranceText()`, the filler and stutter cleanup |
| Cleanup | The stored transcript stays verbatim. `TranscriptView` shows each utterance through `cleanUtteranceText()` while the synced option `transcriptCleanupEnabled` (default on, the "Hide filler words" switch above the transcript) holds, and leaves out utterances it empties. The rules are English-only for stutters (a function-word list) and keep numbers, comma-separated repeats and "that that"/"it it" |
| Speaker names | While a recording writes the transcript, `TranscriptView` renames through `electronApi.transcription.renameSpeaker(attachmentId, …)` (by attachment, since a note can hold an older transcript beside the running one): `AttachmentSink`, the only writer then, applies `setSpeakerName()` to the stored names in one transaction, so no utterance is overwritten. Otherwise it saves the stored transcript whole through the generic `POST notes/:noteId/attachments?matchBy=attachmentId`; attachments get no route of their own. `AttachmentSink` re-reads `speakers` before each rewrite, so a rename from another window or device survives the next utterance |
-| `apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx` | Turns, clock times, the "Speaking…" line, fade-in |
+| `apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx` | Turns, clock times, the "Speaking…" line, fade-in, the level meters |
| `apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx` | The launcher and `useTranscriptionState()` |
## The model
diff --git a/apps/client/src/translations/en/translation.json b/apps/client/src/translations/en/translation.json
index 08fb5cb97c..f1126e066c 100644
--- a/apps/client/src/translations/en/translation.json
+++ b/apps/client/src/translations/en/translation.json
@@ -4604,7 +4604,10 @@
"starting": "Starting the transcription…",
"stop": "Stop transcribing",
"stopping": "Finishing the transcript…",
- "title_keeping_recordings": "{{title}} (keeps the audio for troubleshooting)"
+ "title_keeping_recordings": "{{title}} (keeps the audio for troubleshooting)",
+ "stop_no_audio": "Stop transcribing (no audio is arriving)",
+ "no_audio_microphone": "No audio has arrived from your microphone for 10 seconds, so what you say is not being transcribed. Stop the transcription and start it again.",
+ "no_audio_system": "No audio has arrived from the meeting for 10 seconds, so what the others say is not being transcribed. Stop the transcription and start it again."
},
"transcript": {
"speaker_me": "Me",
@@ -4616,7 +4619,8 @@
"speaking": "Speaking…",
"offset": "{{offset}} into the transcription",
"cleanup": "Hide filler words",
- "cleanup_tooltip": "Hides “uh”, “um” and stuttered words. The transcript itself is not changed."
+ "cleanup_tooltip": "Hides “uh”, “um” and stuttered words. The transcript itself is not changed.",
+ "no_audio": "No audio"
},
"resource_download": {
"title": "Download required",
diff --git a/apps/client/src/widgets/launch_bar/RecordMeetingButton.css b/apps/client/src/widgets/launch_bar/RecordMeetingButton.css
index 3a149ac198..65b6bae7cc 100644
--- a/apps/client/src/widgets/launch_bar/RecordMeetingButton.css
+++ b/apps/client/src/widgets/launch_bar/RecordMeetingButton.css
@@ -31,5 +31,10 @@
color: var(--left-pane-item-selected-color, var(--main-text-color));
box-shadow: var(--left-pane-item-selected-shadow);
font-variant-numeric: tabular-nums;
+
+ &.stalled {
+ --color: var(--danger-color, #dc3545);
+ color: #fff;
+ }
}
}
diff --git a/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsx b/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsx
index f142f14c48..f1342a50f4 100644
--- a/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsx
+++ b/apps/client/src/widgets/launch_bar/RecordMeetingButton.spec.tsx
@@ -294,6 +294,50 @@ describe("RecordMeetingButton", () => {
window.electronApi = undefined;
});
+ it("says once for each source that stops delivering audio, and marks the time", async () => {
+ let onStateChanged: (state: TranscriptionState) => void = () => {};
+ const recording: TranscriptionState = {
+ status: "recording", noteId: "meeting", pendingSources: [], startedAt: Date.now()
+ };
+ const report = (state: Partial<TranscriptionState>) =>
+ act(() => onStateChanged({ ...recording, ...state }));
+ window.electronApi = {
+ transcription: {
+ isSupported: async () => true,
+ getState: async () => recording,
+ onStateChanged: (callback: typeof onStateChanged) => {
+ onStateChanged = callback;
+ return () => {};
+ }
+ }
+ } as unknown as typeof window.electronApi;
+ let container: HTMLElement | null = null;
+ await act(async () => {
+ container = renderInto(<RecordMeetingButton launcherNote={{} as FNote} />);
+ });
+ const badge = () => (container as HTMLElement | null)
+ ?.querySelector(".record-meeting-time-badge");
+ const button = () => (container as HTMLElement | null)?.querySelector("button");
+
+ await vi.waitFor(() => expect(badge()).not.toBeNull());
+ expect(badge()?.classList.contains("stalled")).toBe(false);
+ await report({ stalledSources: [ "system" ] });
+ await report({ stalledSources: [ "system" ] });
+ expect(badge()?.classList.contains("stalled")).toBe(true);
+ expect(button()?.textContent).toContain("record_meeting.stop_no_audio");
+ expect(mocks.showError).toHaveBeenCalledExactlyOnceWith(
+ expect.stringContaining("record_meeting.no_audio_system"));
+
+ await report({});
+ expect(badge()?.classList.contains("stalled")).toBe(false);
+ expect(button()?.textContent).not.toContain("no_audio");
+ await report({ stalledSources: [ "microphone", "system" ] });
+ expect(mocks.showError).toHaveBeenCalledTimes(3);
+ expect(mocks.showError).toHaveBeenCalledWith(
+ expect.stringContaining("record_meeting.no_audio_microphone"));
+ window.electronApi = undefined;
+ });
+
it("shows the shared recording's state, and starts or stops it on a click", async () => {
let onStateChanged: (state: TranscriptionState) => void = () => {};
const report = (status: TranscriptionState["status"]) =>
diff --git a/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx b/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx
index 50beab05a6..59563bbb56 100644
--- a/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx
+++ b/apps/client/src/widgets/launch_bar/RecordMeetingButton.tsx
@@ -2,9 +2,10 @@ import "./RecordMeetingButton.css";
import {
dayjs, getTranscriptionResources, MEETING_RECORDING_LABEL, type TranscriptionState,
- type TranscriptionStatus
+ type TranscriptionStatus, type TranscriptSource
} from "@triliumnext/commons";
-import { useCallback, useEffect, useState } from "preact/hooks";
+import clsx from "clsx";
+import { useCallback, useEffect, useRef, useState } from "preact/hooks";
import dateNoteService from "../../services/date_notes";
import { requireResources } from "../../services/dialog";
@@ -24,11 +25,12 @@ import {
* Launcher button that records a meeting: the first click creates the meeting note and starts the
* recording, the second stops it. Desktop only, since the main process does the recording, and
* shown only while `transcriptionEnabled` is on and the computer can transcribe. Its corner badge reads "Beta" while idle and the
- * time elapsed while recording.
+ * time elapsed while recording, which turns red, with a message once, when a source stops
+ * delivering audio.
*/
export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps) {
const { icon, title } = useLauncherIconAndTitle(launcherNote);
- const { status, error, startedAt } = useTranscriptionState();
+ const { status, error, startedAt, stalledSources } = useTranscriptionState();
const [ isEnabled ] = useTriliumOptionBool("transcriptionEnabled");
const isSupported = useTranscriptionSupported(isEnabled);
const [ keepsRecordings ] = useTriliumOptionBool("transcriptionKeepRecordings");
@@ -43,6 +45,8 @@ export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps)
}
}, [ error ]);
+ useStalledSourceWarnings(stalledSources);
+
useEffect(() => {
if (status !== "recording") {
return;
@@ -62,7 +66,7 @@ export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps)
<LaunchBarActionButton
launcherNote={launcherNote}
onClick={handleClick}
- {...getAppearance(status, icon, title, keepsRecordings)}
+ {...getAppearance(status, icon, title, keepsRecordings, !!stalledSources?.length)}
/>
{status === "idle" && (
<Badge text={t("common.beta")} className="record-meeting-beta-badge" />
@@ -70,13 +74,31 @@ export default function RecordMeetingButton({ launcherNote }: LauncherNoteProps)
{status === "recording" && startedAt !== undefined && (
<Badge
text={formatElapsed(now - startedAt)}
- className="record-meeting-time-badge"
+ className={clsx("record-meeting-time-badge", {
+ stalled: !!stalledSources?.length
+ })}
/>
)}
</div>
);
}
+/** Shows an error once for each source that joins `stalledSources`. */
+function useStalledSourceWarnings(stalledSources: TranscriptSource[] | undefined) {
+ const warned = useRef<TranscriptSource[]>([]);
+ const key = stalledSources?.join() ?? "";
+
+ useEffect(() => {
+ const stalled = stalledSources ?? [];
+ for (const source of stalled.filter((source) => !warned.current.includes(source))) {
+ toast.showError(source === "microphone"
+ ? t("record_meeting.no_audio_microphone")
+ : t("record_meeting.no_audio_system"));
+ }
+ warned.current = stalled;
+ }, [ key ]);
+}
+
const BUSY_ICON = "bx bx-loader-alt bx-spin";
/**
@@ -84,7 +106,8 @@ const BUSY_ICON = "bx bx-loader-alt bx-spin";
* starting is when the people in the meeting would need to agree to it.
*/
function getAppearance(
- status: TranscriptionStatus, idleIcon: string, idleTitle: string, keepsRecordings: boolean
+ status: TranscriptionStatus, idleIcon: string, idleTitle: string, keepsRecordings: boolean,
+ isStalled: boolean
) {
switch (status) {
case "idle":
@@ -99,7 +122,7 @@ function getAppearance(
case "recording":
return {
icon: "bx bx-stop-circle",
- text: t("record_meeting.stop"),
+ text: isStalled ? t("record_meeting.stop_no_audio") : t("record_meeting.stop"),
className: "record-meeting-active"
};
case "stopping":
diff --git a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.css b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.css
index 4f83e8af6c..99411610be 100644
--- a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.css
+++ b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.css
@@ -14,6 +14,52 @@
font-size: 0.9em;
}
+ .transcript-levels {
+ display: flex;
+ flex-wrap: wrap;
+ align-items: center;
+ gap: 0.25rem 1rem;
+ margin-inline-end: auto;
+ user-select: none;
+ }
+
+ .transcript-level {
+ display: flex;
+ align-items: center;
+ gap: 0.35rem;
+
+ .transcript-level-bar {
+ position: relative;
+ width: 4rem;
+ height: 0.35rem;
+ border-radius: 0.2rem;
+ background: color-mix(in srgb, var(--main-text-color) 15%, transparent);
+ overflow: hidden;
+
+ &::after {
+ content: "";
+ position: absolute;
+ inset-block: 0;
+ inset-inline-start: 0;
+ width: calc(var(--level, 0) * 100%);
+ background: var(--main-text-color);
+ transition: width 120ms linear;
+ }
+ }
+
+ &.stalled {
+ color: var(--danger-color, #dc3545);
+
+ .transcript-level-bar {
+ background: var(--danger-color, #dc3545);
+
+ &::after {
+ display: none;
+ }
+ }
+ }
+ }
+
.transcript-turn {
max-width: var(--preferred-max-content-width, 900px);
margin-bottom: 1rem;
@@ -112,4 +158,8 @@
.transcript .transcript-new {
animation: none;
}
+
+ .transcript .transcript-level .transcript-level-bar::after {
+ transition: none;
+ }
}
diff --git a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsx b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsx
index cfc7d147c1..504a38b83f 100644
--- a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsx
+++ b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.spec.tsx
@@ -12,7 +12,7 @@ import options from "../../../services/options";
import { renderInto } from "../../../test/render";
import { ParentComponent } from "../../react/react_utils";
import TranscriptView, {
- formatOffset, groupIntoTurns, type TranscriptTurn
+ formatOffset, groupIntoTurns, levelToMeter, type TranscriptTurn
} from "./TranscriptView";
const h = vi.hoisted(() => ({ post: vi.fn(), put: vi.fn() }));
@@ -113,6 +113,14 @@ describe("transcript helpers", () => {
]);
});
+ it("spreads 60 dB below full scale up to 10 dB below it over a level meter", () => {
+ expect(levelToMeter(0)).toBe(0);
+ expect(levelToMeter(10 ** (-60 / 20))).toBeCloseTo(0);
+ expect(levelToMeter(10 ** (-35 / 20))).toBeCloseTo(0.5);
+ expect(levelToMeter(10 ** (-10 / 20))).toBeCloseTo(1);
+ expect(levelToMeter(1)).toBe(1);
+ });
+
it("formats offsets as m:ss, with hours once there are any", () => {
expect(formatOffset(0)).toBe("0:00");
expect(formatOffset(65_900)).toBe("1:05");
@@ -305,6 +313,7 @@ describe("TranscriptView", () => {
transcription: {
getState: async () => ({ status: "recording", noteId: "meeting", pendingSources: [] }),
onStateChanged: () => () => {},
+ onLevel: () => () => {},
renameSpeaker
}
} as unknown as typeof window.electronApi;
@@ -427,7 +436,8 @@ describe("TranscriptView", () => {
onStateChanged: (callback: typeof onStateChanged) => {
onStateChanged = callback;
return () => {};
- }
+ },
+ onLevel: () => () => {}
}
} as unknown as typeof window.electronApi;
const getBlob = vi.fn()
@@ -465,6 +475,64 @@ describe("TranscriptView", () => {
window.electronApi = undefined;
});
+ it("shows each source's level while its meeting records, and which is silent", async () => {
+ let onStateChanged: (state: TranscriptionState) => void = () => {};
+ let onLevel: (source: TranscriptSource, level: number) => void = () => {};
+ const unsubscribeLevels = vi.fn();
+ const recording: TranscriptionState = {
+ status: "recording", noteId: "meeting", pendingSources: []
+ };
+ const report = (state: Partial<TranscriptionState>) =>
+ act(() => onStateChanged({ ...recording, ...state }));
+ window.electronApi = {
+ transcription: {
+ getState: async () => recording,
+ onStateChanged: (callback: typeof onStateChanged) => {
+ onStateChanged = callback;
+ return () => {};
+ },
+ onLevel: (callback: typeof onLevel) => {
+ onLevel = callback;
+ return unsubscribeLevels;
+ }
+ }
+ } as unknown as typeof window.electronApi;
+ const attachment = {
+ attachmentId: "transcript1", ownerId: "meeting",
+ getBlob: async () => ({ content: transcript([]) })
+ };
+ const container = renderInto(
+ <ParentComponent.Provider value={new Component()}>
+ <TranscriptView attachment={attachment as unknown as FAttachment} />
+ </ParentComponent.Provider>
+ );
+ const meter = (source: string) => container.querySelector<HTMLElement>(
+ `.transcript-level[data-source="${source}"]`);
+ const fill = (source: string) =>
+ Number(meter(source)?.style.getPropertyValue("--level"));
+
+ await vi.waitFor(() => expect(meter("microphone")).not.toBeNull());
+ expect(container.textContent).toContain("transcript.empty");
+ expect(meter("microphone")?.textContent).toContain("transcript.speaker_me");
+ expect(meter("system")?.textContent).toContain("transcript.speaker_others");
+ expect(container.querySelector(".transcript-options input")).toBeNull();
+ act(() => onLevel("microphone", 10 ** (-35 / 20)));
+ expect(fill("microphone")).toBeCloseTo(0.5);
+ expect(fill("system")).toBe(0);
+ expect(meter("microphone")?.getAttribute("aria-valuenow")).toBe("0.5");
+
+ await report({ stalledSources: [ "system" ] });
+ expect(meter("system")?.classList.contains("stalled")).toBe(true);
+ expect(meter("system")?.textContent).toContain("transcript.no_audio");
+ expect(meter("microphone")?.classList.contains("stalled")).toBe(false);
+ expect(meter("microphone")?.textContent).not.toContain("transcript.no_audio");
+
+ await report({ noteId: "other" });
+ expect(meter("microphone")).toBeNull();
+ expect(unsubscribeLevels).toHaveBeenCalledOnce();
+ window.electronApi = undefined;
+ });
+
it("follows new turns for a reader at the end, not for one who scrolled up", async () => {
const utterance = (index: number): TranscriptUtterance =>
({ source: "microphone", offsetMs: index * 60_000, text: `Line ${index}` });
diff --git a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx
index 9ca887c5f1..1eeea630ae 100644
--- a/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx
+++ b/apps/client/src/widgets/type_widgets/transcript/TranscriptView.tsx
@@ -4,6 +4,7 @@ import {
cleanUtteranceText, getSpeakerKey, MAX_SPEAKER_NAME_LENGTH, parseTranscript, setSpeakerName,
type TranscriptSource, type TranscriptUtterance
} from "@triliumnext/commons";
+import clsx from "clsx";
import { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from "preact/hooks";
import type FAttachment from "../../../entities/fattachment";
@@ -31,14 +32,17 @@ const PARAGRAPH_PAUSE_MS = 2000;
* Shows a `transcript` attachment as the conversation it records, one paragraph per speaker turn.
* Follows the attachment as a running recording rewrites it, shows which speakers the recording
* hears before their words are transcribed, and keeps a reader who is at the end there as new
- * turns arrive. With `transcriptCleanupEnabled`, shows each utterance through
+ * turns arrive. While its meeting records, shows each source's level above the turns, and which
+ * source delivers no audio. With `transcriptCleanupEnabled`, shows each utterance through
* `cleanUtteranceText()` and leaves out the ones it empties; the stored transcript stays verbatim.
*/
export default function TranscriptView({ attachment }: { attachment: FAttachment }) {
const [ content, setContent ] = useState<string | null>();
const [ isCleanedUp, setCleanedUp ] = useTriliumOptionBool("transcriptCleanupEnabled");
const recording = useTranscriptionState();
- const pending = recording.noteId === attachment.ownerId ? recording.pendingSources : [];
+ const isOwnRecording = recording.noteId === attachment.ownerId;
+ const pending = isOwnRecording ? recording.pendingSources : [];
+ const isRecording = isOwnRecording && recording.status === "recording";
const scrollerRef = useRef<HTMLDivElement>(null);
/** The scroll height at the previous render, or `null` before the transcript first shows. */
const scrollHeightRef = useRef<number | null>(null);
@@ -86,8 +90,25 @@ export default function TranscriptView({ attachment }: { attachment: FAttachment
if (!transcript) {
return <NoItems icon="bx bx-error" text={t("transcript.unreadable")} />;
}
- if (!transcript.utterances.length && !pending.length) {
- return <NoItems icon="bx bx-microphone" text={t("transcript.empty")} />;
+ const hasUtterances = transcript.utterances.length > 0;
+ const options = (hasUtterances || isRecording) && (
+ <div className="transcript-options">
+ {isRecording && <TranscriptLevels stalledSources={recording.stalledSources ?? []} />}
+ {hasUtterances && (
+ <FormToggle
+ switchOnName={t("transcript.cleanup")}
+ switchOffName={t("transcript.cleanup")}
+ switchOnTooltip={t("transcript.cleanup_tooltip")}
+ switchOffTooltip={t("transcript.cleanup_tooltip")}
+ currentValue={isCleanedUp}
+ onChange={setCleanedUp}
+ />
+ )}
+ </div>
+ );
+ if (!hasUtterances && !pending.length) {
+ const empty = <NoItems icon="bx bx-microphone" text={t("transcript.empty")} />;
+ return options ? <div className="transcript">{options}{empty}</div> : empty;
}
initialCountRef.current ??= transcript.utterances.length;
@@ -110,18 +131,7 @@ export default function TranscriptView({ attachment }: { attachment: FAttachment
);
return (
<div ref={scrollerRef} className="transcript">
- {transcript.utterances.length > 0 && (
- <div className="transcript-options">
- <FormToggle
- switchOnName={t("transcript.cleanup")}
- switchOffName={t("transcript.cleanup")}
- switchOnTooltip={t("transcript.cleanup_tooltip")}
- switchOffTooltip={t("transcript.cleanup_tooltip")}
- currentValue={isCleanedUp}
- onChange={setCleanedUp}
- />
- </div>
- )}
+ {options}
{turns.map((turn) => (
<div key={turn.offsetMs} className={`transcript-turn transcript-${turn.source}`}>
<div className="transcript-turn-header">
@@ -165,6 +175,62 @@ export default function TranscriptView({ attachment }: { attachment: FAttachment
);
}
+const LEVEL_ICONS: Record<TranscriptSource, string> = {
+ microphone: "bx-microphone",
+ system: "bx-volume-full"
+};
+const LEVEL_SOURCES = Object.keys(LEVEL_ICONS) as TranscriptSource[];
+
+/**
+ * One meter per source, named as the transcript names it, filled to its latest level, and saying
+ * so when the source delivers no audio.
+ */
+function TranscriptLevels({ stalledSources }: { stalledSources: TranscriptSource[] }) {
+ const levels = useTranscriptionLevels();
+ return (
+ <div className="transcript-levels">
+ {LEVEL_SOURCES.map((source) => {
+ const level = levelToMeter(levels[source]);
+ const isStalled = stalledSources.includes(source);
+ return (
+ <span
+ key={source}
+ data-source={source}
+ className={clsx("transcript-level", { stalled: isStalled })}
+ role="meter"
+ aria-label={speakerName(source)}
+ aria-valuemin={0}
+ aria-valuemax={1}
+ aria-valuenow={Math.round(level * 100) / 100}
+ style={{ "--level": level }}
+ >
+ <span className={`bx ${LEVEL_ICONS[source]}`} aria-hidden="true" />
+ {speakerName(source)}
+ <span className="transcript-level-bar" aria-hidden="true" />
+ {isStalled && (
+ <span className="transcript-level-warning">
+ {t("transcript.no_audio")}
+ </span>
+ )}
+ </span>
+ );
+ })}
+ </div>
+ );
+}
+
+/** Each source's latest level, from `electronApi.transcription.onLevel()`. */
+function useTranscriptionLevels() {
+ const [ levels, setLevels ] = useState<Record<TranscriptSource, number>>({
+ microphone: 0, system: 0
+ });
+
+ useEffect(() => window.electronApi?.transcription.onLevel((source, level) =>
+ setLevels((current) => ({ ...current, [source]: level }))), []);
+
+ return levels;
+}
+
/**
* A speaker's name, which a click turns into a field that renames every turn of the speaker: Enter
* or leaving the field saves, Escape cancels, and a blank name or the default restores the default.
@@ -324,6 +390,23 @@ export function groupIntoTurns(
}
/** `m:ss`, or `h:mm:ss` from the first hour on. */
+/** The quietest and the loudest RMS a level meter shows, in dB below full scale. */
+const METER_FLOOR_DB = -60;
+const METER_CEILING_DB = -10;
+
+/**
+ * Maps an RMS from 0 to 1 onto a level meter's 0 to 1, in decibels: speech, about 35 dB below full
+ * scale, fills half of it.
+ */
+export function levelToMeter(level: number) {
+ if (level <= 0) {
+ return 0;
+ }
+ const db = 20 * Math.log10(level);
+ const fraction = (db - METER_FLOOR_DB) / (METER_CEILING_DB - METER_FLOOR_DB);
+ return Math.min(1, Math.max(0, fraction));
+}
+
export function formatOffset(milliseconds: number) {
const totalSeconds = Math.max(0, Math.floor(milliseconds / 1000));
const hours = Math.floor(totalSeconds / 3600);
diff --git a/apps/desktop/src/preload.spec.ts b/apps/desktop/src/preload.spec.ts
index 19ea491a2b..1f2ab389f8 100644
--- a/apps/desktop/src/preload.spec.ts
+++ b/apps/desktop/src/preload.spec.ts
@@ -756,5 +756,18 @@ describe("preload script", () => {
unsubscribe();
expect(ipcRendererListeners.has("transcription-state")).toBe(false);
});
+
+ it("onLevel relays each source's level until unsubscribed", () => {
+ const callback = vi.fn();
+ const unsubscribe = transcription().onLevel(callback);
+
+ const listeners = ipcRendererListeners.get("transcription-level") ?? [];
+ expect(listeners).toHaveLength(1);
+ listeners[0]({}, "microphone", 0.25);
+ expect(callback).toHaveBeenCalledExactlyOnceWith("microphone", 0.25);
+
+ unsubscribe();
+ expect(ipcRendererListeners.has("transcription-level")).toBe(false);
+ });
});
});
diff --git a/apps/desktop/src/preload.ts b/apps/desktop/src/preload.ts
index fdd2f49d6d..387f931101 100644
--- a/apps/desktop/src/preload.ts
+++ b/apps/desktop/src/preload.ts
@@ -6,7 +6,8 @@ import type {
NativeImportOptions,
OneNoteLoginResult,
RendererStartupMetric,
- TranscriptionState
+ TranscriptionState,
+ TranscriptSource
} from "@triliumnext/commons";
import { contextBridge, ipcRenderer, webFrame, webUtils } from "electron";
@@ -277,6 +278,12 @@ contextBridge.exposeInMainWorld("electronApi", {
const listener = (_event: unknown, state: TranscriptionState) => callback(state);
ipcRenderer.on("transcription-state", listener);
return () => ipcRenderer.removeListener("transcription-state", listener);
+ },
+ onLevel(callback: (source: TranscriptSource, level: number) => void) {
+ const listener = (_event: unknown, source: TranscriptSource, level: number) =>
+ callback(source, level);
+ ipcRenderer.on("transcription-level", listener);
+ return () => ipcRenderer.removeListener("transcription-level", listener);
}
},
diff --git a/apps/desktop/src/services/transcription/index.spec.ts b/apps/desktop/src/services/transcription/index.spec.ts
index dd1b06fc5f..d9d353a374 100644
--- a/apps/desktop/src/services/transcription/index.spec.ts
+++ b/apps/desktop/src/services/transcription/index.spec.ts
@@ -1,6 +1,6 @@
import type { TranscriptionState } from "@triliumnext/commons";
import path from "path";
-import { beforeEach, describe, expect, it, vi } from "vitest";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { CaptureWindowOptions } from "./capture";
import type { WorkerListeners } from "./worker_client";
@@ -10,6 +10,7 @@ const h = vi.hoisted(() => ({
appListeners: new Map<string, (...args: unknown[]) => unknown>(),
quit: vi.fn(),
broadcast: [] as TranscriptionState[],
+ levels: [] as [ string, number ][],
notes: new Set<string>([ "meeting" ]),
isProtected: false,
isProtectedSessionAvailable: false,
@@ -42,9 +43,11 @@ vi.mock("electron", () => ({
BrowserWindow: {
getAllWindows: () => [ 1, 2 ].map(() => ({
webContents: {
- send: (channel: string, state: TranscriptionState) => {
+ send: (channel: string, ...args: unknown[]) => {
if (channel === "transcription-state") {
- h.broadcast.push(state);
+ h.broadcast.push(args[0] as TranscriptionState);
+ } else if (channel === "transcription-level") {
+ h.levels.push(args as [ string, number ]);
}
}
}
@@ -142,6 +145,7 @@ beforeEach(() => {
h.isProtected = false;
h.isProtectedSessionAvailable = false;
h.broadcast.length = 0;
+ h.levels.length = 0;
h.keepRecordings = false;
h.transcriptionModel = null;
h.speakersEnabled = true;
@@ -155,6 +159,10 @@ beforeEach(() => {
h.capture.stop.mockResolvedValue(undefined);
});
+afterEach(() => {
+ vi.useRealTimers();
+});
+
describe("transcription IPC", () => {
it("records one meeting at a time and tells every window about each step", async () => {
const invoke = await loadService();
@@ -239,6 +247,49 @@ describe("transcription IPC", () => {
expect(h.broadcast.at(-1)).toEqual(IDLE);
});
+ it("passes each source's level on to every window", async () => {
+ const invoke = await loadService();
+ await invoke("transcription-start", "meeting");
+ h.listeners?.onLevel("system", 0.125);
+ expect(h.levels).toEqual([ [ "system", 0.125 ], [ "system", 0.125 ] ]);
+ });
+
+ it("flags a source no audio has come from for 10 s, until it comes again", async () => {
+ const invoke = await loadService();
+ vi.useFakeTimers();
+ await invoke("transcription-start", "meeting");
+ const recording = {
+ status: "recording", noteId: "meeting", pendingSources: [], startedAt: STARTED_AT
+ };
+ const tick = async () => {
+ await vi.advanceTimersByTimeAsync(1000);
+ h.listeners?.onLevel("system", 0.1);
+ };
+
+ for (let second = 1; second < 10; second++) {
+ await tick();
+ }
+ expect(h.broadcast.at(-1)).toEqual(recording);
+ await tick();
+ expect(h.broadcast.at(-1)).toEqual({ ...recording, stalledSources: [ "microphone" ] });
+ expect(await invoke("transcription-get-state")).toEqual(h.broadcast.at(-1));
+ expect(h.log.error).toHaveBeenCalledExactlyOnceWith(
+ "Transcription: no audio from the microphone for 10 s.");
+ const flagged = h.broadcast.length;
+ await tick();
+ expect(h.broadcast).toHaveLength(flagged);
+
+ h.listeners?.onLevel("microphone", 0);
+ expect(h.broadcast.at(-1)).toEqual(recording);
+ expect(h.log.info).toHaveBeenCalledWith("Transcription: audio from the microphone again.");
+
+ await invoke("transcription-stop");
+ const stopped = h.broadcast.length;
+ await vi.advanceTimersByTimeAsync(20_000);
+ expect(h.broadcast).toHaveLength(stopped);
+ vi.useRealTimers();
+ });
+
it("says once that a locked protected session holds back the transcript, and saves it on unlocking", async () => {
h.isProtected = true;
const invoke = await loadService();
diff --git a/apps/desktop/src/services/transcription/index.ts b/apps/desktop/src/services/transcription/index.ts
index 93d365ee64..0e4082db56 100644
--- a/apps/desktop/src/services/transcription/index.ts
+++ b/apps/desktop/src/services/transcription/index.ts
@@ -1,4 +1,6 @@
-import type { TranscriptionState, TranscriptUtterance } from "@triliumnext/commons";
+import type {
+ TranscriptionState, TranscriptSource, TranscriptUtterance
+} from "@triliumnext/commons";
import { becca, events, getLog, options, protected_session } from "@triliumnext/core";
import dataDirs from "@triliumnext/server/src/services/data_dir.js";
import resources from "@triliumnext/server/src/services/resources.js";
@@ -23,9 +25,18 @@ const IDLE: TranscriptionState = { status: "idle", noteId: null, pendingSources:
/** How long quitting waits for a running transcription to finish. */
const QUIT_TIMEOUT_MS = 15_000;
+/**
+ * How long a source can go without delivering a chunk before it counts as stalled. A silent
+ * source still delivers chunks of silence, about four a second.
+ */
+const NO_AUDIO_MS = 10_000;
+
interface Recording {
worker: TranscriptionWorker;
capture: CaptureWindow;
+ /** When each source last delivered a chunk, in milliseconds since the epoch. */
+ lastHeard: Map<TranscriptSource, number>;
+ audioCheck?: ReturnType<typeof setInterval>;
}
let state = IDLE;
@@ -87,9 +98,11 @@ export async function startRecording(noteId: unknown): Promise<string> {
worker: new TranscriptionWorker({
onUtterance: (utterance) => writeUtterance(sink, noteId, utterance),
onPendingChanged: (pendingSources) => setState({ ...state, pendingSources }),
+ onLevel: (source, level) => handleLevel(recording, source, level),
onUnexpectedExit: (code) => handleUnexpectedExit(recording, code),
log: getLog()
- })
+ }),
+ lastHeard: new Map()
};
activeRecording = recording;
try {
@@ -106,11 +119,16 @@ export async function startRecording(noteId: unknown): Promise<string> {
recording.worker.connectCapture(port2);
const sources = await recording.capture.start(port1);
getLog().info(`Transcription: recording ${sources.join(" and ")} for note ${noteId}.`);
+ for (const source of sources) {
+ recording.lastHeard.set(source, Date.now());
+ }
+ recording.audioCheck = setInterval(() => checkAudio(recording), 1000);
setState({ ...state, status: "recording", startedAt });
return "";
} catch (e) {
activeRecording = null;
writer = null;
+ clearInterval(recording.audioCheck);
void recording.capture.stop();
recording.worker.kill();
setState(IDLE);
@@ -131,6 +149,7 @@ export function stopRecording(error?: string): Promise<void> {
setState({ ...state, status: "stopping" });
activeRecording = null;
+ clearInterval(recording.audioCheck);
const { noteId } = state;
stopping = recording.capture.stop()
.then(() => recording.worker.stop())
@@ -273,12 +292,53 @@ function handleUnexpectedExit(recording: Recording, code: number) {
}
activeRecording = null;
writer = null;
+ clearInterval(recording.audioCheck);
void recording.capture.stop();
getLog().error(`Transcription: the transcription process exited with code ${code}.`);
state = IDLE;
broadcast({ ...IDLE, error: t("transcription.stopped-unexpectedly") });
}
+/** Passes a source's level on to every window, and clears that source's stall. */
+function handleLevel(recording: Recording, source: TranscriptSource, level: number) {
+ if (activeRecording !== recording) {
+ return;
+ }
+ recording.lastHeard.set(source, Date.now());
+ if (state.stalledSources?.includes(source)) {
+ getLog().info(`Transcription: audio from the ${source} again.`);
+ setStalledSources(state.stalledSources.filter((stalled) => stalled !== source));
+ }
+ for (const window of BrowserWindow.getAllWindows()) {
+ window.webContents.send("transcription-level", source, level);
+ }
+}
+
+/** Reports the sources that have delivered nothing for {@link NO_AUDIO_MS}. */
+function checkAudio(recording: Recording) {
+ if (activeRecording !== recording || state.status !== "recording") {
+ return;
+ }
+ const now = Date.now();
+ const stalled = [ ...recording.lastHeard ]
+ .filter(([ , lastHeard ]) => now - lastHeard >= NO_AUDIO_MS)
+ .map(([ source ]) => source)
+ .sort();
+ const current = state.stalledSources ?? [];
+ if (stalled.join() === current.join()) {
+ return;
+ }
+ for (const source of stalled.filter((source) => !current.includes(source))) {
+ getLog().error(`Transcription: no audio from the ${source} for ${NO_AUDIO_MS / 1000} s.`);
+ }
+ setStalledSources(stalled);
+}
+
+function setStalledSources(stalledSources: TranscriptSource[]) {
+ const { stalledSources: _, ...rest } = state;
+ setState(stalledSources.length ? { ...rest, stalledSources } : rest);
+}
+
function setState(newState: TranscriptionState) {
state = newState;
broadcast(state);
diff --git a/apps/desktop/src/services/transcription/protocol.ts b/apps/desktop/src/services/transcription/protocol.ts
index 607b819761..a16dd4ff47 100644
--- a/apps/desktop/src/services/transcription/protocol.ts
+++ b/apps/desktop/src/services/transcription/protocol.ts
@@ -41,6 +41,8 @@ export type WorkerEvent =
| { type: "failed"; message: string }
| { type: "pending"; sources: TranscriptSource[] }
| { type: "utterance"; utterance: TranscriptUtterance }
+ /** The RMS of a source's latest chunk, from 0 to 1; one per chunk, about four a second. */
+ | { type: "level"; source: TranscriptSource; level: number }
| { type: "log"; level: keyof Logger; message: string }
/** Every utterance captured before `stop` has been sent. */
| { type: "stopped" }
diff --git a/apps/desktop/src/services/transcription/session.spec.ts b/apps/desktop/src/services/transcription/session.spec.ts
index 4f8efe7b99..6c7c9b7131 100644
--- a/apps/desktop/src/services/transcription/session.spec.ts
+++ b/apps/desktop/src/services/transcription/session.spec.ts
@@ -80,12 +80,13 @@ function createHarness(extraSinks: TranscriptSink[] = [], taps: AudioTap[] = [])
};
const sinks = [ ...extraSinks, sink ];
const onPendingChanged = vi.fn();
+ const onLevel = vi.fn();
const session = new RecordingSession({
- capture, engine, sinks, taps, log, onPendingChanged, now: () => clock
+ capture, engine, sinks, taps, log, onPendingChanged, onLevel, now: () => clock
});
return {
- session, capture, engine, sink, written, flushed, onPendingChanged,
+ session, capture, engine, sink, written, flushed, onPendingChanged, onLevel,
/** Delivers `samples` from `source` when the clock reads `at`. */
deliver(source: TranscriptSource, samples: Float32Array, at: number) {
clock = at;
@@ -276,6 +277,22 @@ describe("RecordingSession", () => {
await session.stop();
});
+ it("reports the loudness of every chunk as it arrives, per source", async () => {
+ const { session, deliver, onLevel } = createHarness();
+ await session.start();
+ const loud = speechChunk(true).fill(0.5, 3);
+ deliver("microphone", speechChunk(false), 2000);
+ deliver("system", loud, 3000);
+
+ expect(onLevel).toHaveBeenCalledTimes(2);
+ const [ [ quietSource, quiet ], [ loudSource, level ] ] = onLevel.mock.calls;
+ expect(quietSource).toBe("microphone");
+ expect(quiet).toBeCloseTo(Math.sqrt(1 / 16000));
+ expect(loudSource).toBe("system");
+ expect(level).toBeCloseTo(Math.sqrt((2 + 0.25 * 15997) / 16000));
+ await session.stop();
+ });
+
it("logs a failed transcription or tap, and still finishes the rest", async () => {
const tap = { write: vi.fn(), close: vi.fn(() => {
throw new Error("disk full");
diff --git a/apps/desktop/src/services/transcription/session.ts b/apps/desktop/src/services/transcription/session.ts
index 22b8c6a3ca..ed5f4dbb5d 100644
--- a/apps/desktop/src/services/transcription/session.ts
+++ b/apps/desktop/src/services/transcription/session.ts
@@ -53,6 +53,8 @@ export interface RecordingSessionOptions {
* still being spoken or because it waits to be transcribed, every time that list changes.
*/
onPendingChanged?: (sources: TranscriptSource[]) => void;
+ /** Called with the RMS of every chunk as it arrives, from 0 to 1. */
+ onLevel?: (source: TranscriptSource, level: number) => void;
now?: () => number;
}
@@ -87,7 +89,7 @@ export class RecordingSession {
try {
const pipeline = this.pipelineFor(source, samples);
this.tap(source, samples, pipeline.firstSampleOffsetMs);
- pipeline.accept(samples);
+ this.options.onLevel?.(source, pipeline.accept(samples));
this.updatePending();
} catch (e) {
this.log.error(`Transcription: ${source} failed: ${(e as Error)?.message ?? e}`);
@@ -228,7 +230,10 @@ export class RecordingSession {
}
}
-/** Segments one source and logs its peak level every 5 s, to spot a silent source. */
+/**
+ * Segments one source, measures each chunk's RMS and logs the source's peak level every 5 s, to
+ * spot a silent source.
+ */
class SourcePipeline {
private peak = 0;
private peakSampleCount = 0;
@@ -242,9 +247,11 @@ class SourcePipeline {
private readonly log: Logger
) {}
+ /** Segments `samples` and returns their RMS. */
accept(samples: Float32Array) {
- this.trackPeak(samples);
+ const level = this.measure(samples);
this.emit(this.segmenter.accept(samples));
+ return level;
}
flush() {
@@ -262,9 +269,11 @@ class SourcePipeline {
}
}
- private trackPeak(samples: Float32Array) {
+ private measure(samples: Float32Array) {
+ let sumOfSquares = 0;
for (const sample of samples) {
this.peak = Math.max(this.peak, Math.abs(sample));
+ sumOfSquares += sample * sample;
}
this.peakSampleCount += samples.length;
if (this.peakSampleCount >= 5 * SAMPLE_RATE) {
@@ -272,6 +281,7 @@ class SourcePipeline {
this.peak = 0;
this.peakSampleCount = 0;
}
+ return samples.length ? Math.sqrt(sumOfSquares / samples.length) : 0;
}
}
diff --git a/apps/desktop/src/services/transcription/transcription_worker.ts b/apps/desktop/src/services/transcription/transcription_worker.ts
index 3e3e3c32b9..03a0a7cdf3 100644
--- a/apps/desktop/src/services/transcription/transcription_worker.ts
+++ b/apps/desktop/src/services/transcription/transcription_worker.ts
@@ -65,7 +65,8 @@ async function start({ noteId, models, recordingDirectory }: StartRequest) {
sinks,
taps,
log,
- onPendingChanged: (sources) => post({ type: "pending", sources })
+ onPendingChanged: (sources) => post({ type: "pending", sources }),
+ onLevel: (source, level) => post({ type: "level", source, level })
});
await newSession.start();
session = newSession;
diff --git a/apps/desktop/src/services/transcription/worker_client.spec.ts b/apps/desktop/src/services/transcription/worker_client.spec.ts
index ef44414fdd..d4913a9bcf 100644
--- a/apps/desktop/src/services/transcription/worker_client.spec.ts
+++ b/apps/desktop/src/services/transcription/worker_client.spec.ts
@@ -44,6 +44,7 @@ beforeEach(() => {
listeners = {
onUtterance: vi.fn(),
onPendingChanged: vi.fn(),
+ onLevel: vi.fn(),
onUnexpectedExit: vi.fn(),
log: { info: vi.fn(), error: vi.fn() }
};
@@ -96,6 +97,8 @@ describe("TranscriptionWorker", () => {
child.report({ type: "utterance", utterance });
child.report({ type: "pending", sources: [ "system" ] });
child.report({ type: "log", level: "error", message: "system failed" });
+ child.report({ type: "level", source: "microphone", level: 0.25 });
+ expect(listeners.onLevel).toHaveBeenCalledExactlyOnceWith("microphone", 0.25);
expect(listeners.onUtterance).toHaveBeenCalledExactlyOnceWith(utterance);
expect(listeners.onPendingChanged).toHaveBeenCalledExactlyOnceWith([ "system" ]);
expect(listeners.log.error).toHaveBeenCalledExactlyOnceWith("system failed");
diff --git a/apps/desktop/src/services/transcription/worker_client.ts b/apps/desktop/src/services/transcription/worker_client.ts
index ee237bc48b..b820b045ff 100644
--- a/apps/desktop/src/services/transcription/worker_client.ts
+++ b/apps/desktop/src/services/transcription/worker_client.ts
@@ -8,6 +8,8 @@ import type { Logger, WorkerEvent, WorkerRequest } from "./protocol";
export interface WorkerListeners {
onUtterance(utterance: TranscriptUtterance): void;
onPendingChanged(sources: TranscriptSource[]): void;
+ /** Called with the RMS of each chunk a source delivers, from 0 to 1. */
+ onLevel(source: TranscriptSource, level: number): void;
/** Called when the process ends without having been stopped, as when it crashes. */
onUnexpectedExit(code: number): void;
log: Logger;
@@ -94,6 +96,9 @@ export class TranscriptionWorker {
case "utterance":
this.listeners.onUtterance(event.utterance);
break;
+ case "level":
+ this.listeners.onLevel(event.source, event.level);
+ break;
case "log":
this.listeners.log[event.level](event.message);
break;
diff --git a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.html b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.html
index f86e19d675..3cb8fad848 100644
--- a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.html
+++ b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Meeting transcription.html
@@ -45,6 +45,11 @@
<p>Only one meeting can be transcribed at a time. The note itself is an ordinary
text note: you can take notes in it during the meeting, rename it or move
it anywhere in the tree. It carries the <code>#meetingRecording</code> label.</p>
+<p>If no audio arrives from your microphone or from the meeting for 10 seconds,
+ Trilium shows a message, the time on the button turns red and the button's
+ tooltip says that no audio is arriving. Silence does not cause this: a
+ muted microphone still delivers audio. Stop the transcription and start
+ it again; what was transcribed until then is kept.</p>
<p>If the transcription stops on its own, for example because the audio could
no longer be captured, Trilium shows a message and keeps everything transcribed
until then. Closing Trilium during a meeting waits for the transcription
@@ -53,9 +58,13 @@
<p>The transcript is stored as an <a href="#root/_help_0vhv7lsOLy82">attachment</a> of
the meeting note and is shown in a collapsible <em>Transcript</em> section
above the note's content. It updates as the meeting goes on, one paragraph
- per speaker turn, each with the time it started. While someone is talking, <em>Speaking…</em> shows
- under their name until their words are transcribed. If you are reading
- the end of the transcript, it scrolls to follow new text.</p>
+ per speaker turn, each with the time it started. While the meeting is being
+ transcribed, a meter above the transcript shows the level of your microphone
+ (<em>Me</em>) and of the meeting's audio (<em>Others</em>), so you can
+ tell that both are being heard; a source that delivers no audio is marked <em>No audio</em>.
+ While someone is talking, <em>Speaking…</em> shows under their name until
+ their words are transcribed. If you are reading the end of the transcript,
+ it scrolls to follow new text.</p>
<ul>
<li>What you say into your microphone is labeled <em>Me</em>.</li>
<li>What the other participants say is labeled <em>Speaker 1</em>, <em>Speaker 2</em> and
diff --git a/docs/User Guide/User Guide/Meeting transcription.md b/docs/User Guide/User Guide/Meeting transcription.md
index 8fcd777745..f1263014cf 100644
--- a/docs/User Guide/User Guide/Meeting transcription.md
+++ b/docs/User Guide/User Guide/Meeting transcription.md
@@ -25,11 +25,13 @@ The speech recognition models are not part of Trilium. The first time you transc
Only one meeting can be transcribed at a time. The note itself is an ordinary text note: you can take notes in it during the meeting, rename it or move it anywhere in the tree. It carries the `#meetingRecording` label.
+If no audio arrives from your microphone or from the meeting for 10 seconds, Trilium shows a message, the time on the button turns red and the button's tooltip says that no audio is arriving. Silence does not cause this: a muted microphone still delivers audio. Stop the transcription and start it again; what was transcribed until then is kept.
+
If the transcription stops on its own, for example because the audio could no longer be captured, Trilium shows a message and keeps everything transcribed until then. Closing Trilium during a meeting waits for the transcription to finish first.
## The transcript
-The transcript is stored as an [attachment](Basic%20Concepts%20and%20Features/Notes/Attachments.md) of the meeting note and is shown in a collapsible _Transcript_ section above the note's content. It updates as the meeting goes on, one paragraph per speaker turn, each with the time it started. While someone is talking, _Speaking…_ shows under their name until their words are transcribed. If you are reading the end of the transcript, it scrolls to follow new text.
+The transcript is stored as an [attachment](Basic%20Concepts%20and%20Features/Notes/Attachments.md) of the meeting note and is shown in a collapsible _Transcript_ section above the note's content. It updates as the meeting goes on, one paragraph per speaker turn, each with the time it started. While the meeting is being transcribed, a meter above the transcript shows the level of your microphone (_Me_) and of the meeting's audio (_Others_), so you can tell that both are being heard; a source that delivers no audio is marked _No audio_. While someone is talking, _Speaking…_ shows under their name until their words are transcribed. If you are reading the end of the transcript, it scrolls to follow new text.
* What you say into your microphone is labeled _Me_.
* What the other participants say is labeled _Speaker 1_, _Speaker 2_ and so on, while _Identify speakers_ is on; otherwise it is labeled _Others_.
diff --git a/packages/commons/src/lib/electron_api_interface.ts b/packages/commons/src/lib/electron_api_interface.ts
index 34b0f5b70b..303402bb24 100644
--- a/packages/commons/src/lib/electron_api_interface.ts
+++ b/packages/commons/src/lib/electron_api_interface.ts
@@ -724,6 +724,11 @@ export interface TranscriptionState {
noteId: string | null;
/** The sources heard speaking, or whose speech is still being transcribed. */
pendingSources: TranscriptSource[];
+ /**
+ * The sources that have delivered no audio for 10 s while recording, as when their capture
+ * hangs; absent when every source delivers. A muted source still delivers silence.
+ */
+ stalledSources?: TranscriptSource[];
/**
* When the recording started, in milliseconds since the epoch: set once it records, kept while
* it stops, absent otherwise.
@@ -780,6 +785,14 @@ export interface ElectronTranscriptionApi {
* @returns A function that removes the listener.
*/
onStateChanged(callback: (state: TranscriptionState) => void): () => void;
+
+ /**
+ * Calls `callback` with the RMS, from 0 to 1, of every chunk a source of the running recording
+ * delivers, about four times a second per source.
+ *
+ * @returns A function that stops the calls.
+ */
+ onLevel(callback: (source: TranscriptSource, level: number) => void): () => void;
}
/**