Skip to content

Voice-to-text dictation app with local (Nvidia Parakeet/Whisper) and cloud models (BYOK). Privacy-first and available cross-platform.

fix: save uploaded transcription in the transcript field (#2880)

十方空烬 · Ember ☄️16bd34971ecc
diff --git a/CLAUDE.md b/CLAUDE.md
index 6a9cf2719..d2d653823 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -788,7 +788,7 @@ Rows in the `actions` table (built-ins seeded from `src/helpers/builtinActions.j
- **Templates** (`kind: 'template'`) write a note's AI summary (`enhanced_content`) from its notes, meeting context, and transcript, never from the summary they replace. A template is a context (`prompt`) plus ordered `sections` (`{heading, instruction}`); `compileTemplatePrompt` wraps sections in the shared notes rules. A template without sections keeps the request it always had after the `SOURCE_FIDELITY_RULE` prefix (the generic system prompt, or the material preamble for a standalone key). The note records the template in `enhancement_template_id` (its `client_id`), which syncs with the note, and `enhancement_prompt` the compiled system prompt
- **Actions** (`kind: 'action'`) work from the AI summary when the note has one, and from its notes and transcript otherwise (`buildNoteRunInput`). Only the built-in Make notes shorter (`shorten`) and Make notes longer (`lengthen`) actions edit that summary, selected by fixed built-in identity (`toActionItem`), never an editable name or stored `output`. A summary edit requires complete output, never splits into parts, and writes only `enhanced_content`; with no summary yet it writes one from the transcript and applies the action to it, run like a template (split into parts on a local model, the material hash recorded); every other built-in and custom action, including Write TL;DR (`addTldr`), runs in the note chat via `SendToAIOptions.requestText` (the chat shows a built-in's call to action where it has one, otherwise the action's name, and the preamble points the model at the summary or the transcript and has it answer in the chat as text in the note's language, using a tool to draft, send or post only when the action's instructions explicitly ask to send, post or file something, and leaving the note unchanged, with an asked-for edit written in the chat instead; its turn is never offered `update_note` (`keepNotesUnchanged`), so it can't change a note whatever the model decides; the note chat's other tools and attendee block otherwise apply as usual, and its transcript context names speakers only on notes known to be the user's own (`ownsNote`), never listing the unfiltered invitees)
-- **The material picks the wrapper** (`runNoteAction`): speaker segments make a meeting (`isMeetingNote`: `MEETING_SYSTEM_PROMPT` or `MEETING_INPUT_PREAMBLE`, the `## Meeting Context` block and `Speaker: text` lines); a transcript in plain text (a recording that captured no segments, one made before 1.6.6, or an upload's text when its provider returned no segments, which is saved as the note's content) is its own case (`hasPlainTranscript`: `PLAIN_TRANSCRIPT_SYSTEM_PROMPT` or `PLAIN_TRANSCRIPT_INPUT_PREAMBLE`, #2666), with no Meeting Context since nothing shows who is speaking, a part prompt without speaker labels when a local model splits it, and action items as `- [ ] Action — Owner` lines with an owner or deadline only when the transcript or the user's notes state it, never a placeholder, and never a table unless the instructions ask for one; no transcript is a personal note (`BASE_SYSTEM_PROMPT` or `NOTE_INPUT_PREAMBLE`). Both transcript cases get the custom dictionary, and their known people (the user, the invitees, and a meeting's named speakers) turn matching owners into mentions (`tagActionItemOwners`)
+- **The material picks the wrapper** (`runNoteAction`): speaker segments make a meeting (`isMeetingNote`: `MEETING_SYSTEM_PROMPT` or `MEETING_INPUT_PREAMBLE`, the `## Meeting Context` block and `Speaker: text` lines); a transcript in plain text (a recording that captured no segments, one made before 1.6.6, or an upload whose provider named no speakers: its text is stored as segments with no speaker or source (`speakerlessTranscriptText`; one untimed segment when it returned no segments), or as the note's content on uploads saved before the transcript held it) is its own case (`hasPlainTranscript`: `PLAIN_TRANSCRIPT_SYSTEM_PROMPT` or `PLAIN_TRANSCRIPT_INPUT_PREAMBLE`, #2666), with no Meeting Context since nothing shows who is speaking, a part prompt without speaker labels when a local model splits it, and action items as `- [ ] Action — Owner` lines with an owner or deadline only when the transcript or the user's notes state it, never a placeholder, and never a table unless the instructions ask for one; no transcript is a personal note (`BASE_SYSTEM_PROMPT` or `NOTE_INPUT_PREAMBLE`). Both transcript cases get the custom dictionary, and their known people (the user, the invitees, and a meeting's named speakers) turn matching owners into mentions (`tagActionItemOwners`)
- The default template is the built-in AI Summary (the Detailed Notes sections, stored under the old Generate Notes key `generateNotes`), listed first (launch re-applies the built-in order). Launch retires a stock Detailed Notes row (`LEGACY_DETAILED_NOTES`) unless AI Summary was customized; an edited one is kept and listed second, and saved template ids are never rewritten. Invariant: AI Summary, compiled from its sections, equals the flat 1.10.2 Detailed Notes prompt it replaced after the `SOURCE_FIDELITY_RULE` prefix used by templates and summary actions (`sourceFidelity.js`), but for the closing rule on material with nothing to summarize (`test/helpers/templatePrompts.test.js`); a built-in's `previousPrompts` only upgrades rows that are still flat. Chat actions get the rule once, from the chat system prompt (`getAgentSystemPrompt`), and the Cloud request leaves out the leading system message that repeats `systemPrompt`, which the API prepends itself
- For material with nothing substantive (only greetings, filler or recording checks), sectioned templates, AI Summary and a summary action writing a first summary ask the model to answer `NOTHING_TO_SUMMARIZE`; custom context-only templates and flat built-ins edited before this rule keep their own rules. A reply that is the marker (bare, fenced, emphasised or after a short sentence) outside a thinking block writes nothing; a summary that merely contains it is saved, and the runner shows `notes.actions.errors.nothingToSummarize` as a neutral notice (`notice` on the error event) instead of saving a placeholder summary
- UI: `TemplatePicker` opens from the chevron in the AI Summary tab, which names the note's template; the "Generate AI Summary" callout (runs the note's template, else the default), `ActionPicker` in the collapsed ask bar (runs the last action picked from it, in place of the send button), `ActionChips` once the in-view chat opens and in the docked chat (the first five actions in docked chat; the centered row omits Make notes longer without backfilling another chip; each has its action's icon, and All actions keeps every action; hovering a chip shows its icon, title and description without a destination badge; the in-view chat pins All actions at the end, and the docked chat shows them as filled pills in one tray with its composer: All actions, then the actions, scrolling together), a `/` menu in the note composers (`slashCommands` on `ChatInput`, each row with its action's icon, or a "/" for an icon this build doesn't know; it fills the chat above the composer until a command runs, the "/" is deleted, nothing matches or the composer loses focus), and `ActionManagerDialog` (Templates | Actions). The notes list and the docked chat widen or narrow from their edge facing the note (`useResizableWidth`, `PanelResizeHandle`), and keep their width across launches (localStorage `notesSidebarWidth`, `noteChatWidth`). Anything the ask bar sends (a message, or a chat action from the picker, a chip or the `/` menu) starts a new conversation in the docked chat, which holds the conversation and its history; the in-view chat is only the chips and the composer. Destinations are fixed and described in the editor; saving never asks a model to choose one. Legacy custom summary destinations read as chat (`toActionItem`), keeping the existing IPC fields, IDs and prompts. Summary generation stays in the AI Summary tab and callout. A run that writes the summary is undoable for 6 s instead of asking first: the runner queues the fields it overwrote (`appliedEvents`) and `BackgroundActionToastListener` shows the Undo toast
diff --git a/openwhispr-mobile/src/lib/notes/__tests__/noteBodyTabs.test.ts b/openwhispr-mobile/src/lib/notes/__tests__/noteBodyTabs.test.ts
index 5ea5d4b09..d7e313297 100644
--- a/openwhispr-mobile/src/lib/notes/__tests__/noteBodyTabs.test.ts
+++ b/openwhispr-mobile/src/lib/notes/__tests__/noteBodyTabs.test.ts
@@ -7,6 +7,7 @@ import {
} from '@/lib/notes/noteBodyTabs';
const tabInput = (overrides: Partial<NoteBodyTabInput> = {}): NoteBodyTabInput => ({
+ isUpload: false,
usesSegmentTranscript: false,
transcriptPending: false,
contentIsTranscript: false,
@@ -44,7 +45,7 @@ describe('getNoteBodyTabs', () => {
});
it('never shows an upload body beside its transcript', () => {
- const upload = { contentIsTranscript: true };
+ const upload = { isUpload: true, contentIsTranscript: true };
expect(getNoteBodyTabs(tabInput({ ...upload, usesSegmentTranscript: true }))).toEqual([
'transcript',
]);
@@ -56,6 +57,15 @@ describe('getNoteBodyTabs', () => {
]);
});
+ it('keeps an upload transcript beside its own notes before and after generation', () => {
+ const upload = tabInput({ isUpload: true, usesSegmentTranscript: true });
+ expect(getNoteBodyTabs(upload)).toEqual(['transcript', 'notes']);
+ const generated = { ...upload, hasEnhanced: true };
+ expect(getNoteBodyTabs(generated)).toEqual(['enhanced', 'transcript', 'notes']);
+ expect(resolveNoteBodyView('transcript', generated)).toBe('transcript');
+ expect(resolveNoteBodyView('notes', generated)).toBe('notes');
+ });
+
it('shows an upload without segments as its editable body', () => {
expect(getNoteBodyTabs(tabInput({ contentIsTranscript: true }))).toEqual(['notes']);
expect(getNoteBodyTabs(tabInput({ contentIsTranscript: true, hasEnhanced: true }))).toEqual([
diff --git a/openwhispr-mobile/src/lib/notes/noteBodyTabs.ts b/openwhispr-mobile/src/lib/notes/noteBodyTabs.ts
index af0fb65cc..4b6cc2d0c 100644
--- a/openwhispr-mobile/src/lib/notes/noteBodyTabs.ts
+++ b/openwhispr-mobile/src/lib/notes/noteBodyTabs.ts
@@ -1,10 +1,11 @@
export type NoteBodyView = 'enhanced' | 'transcript' | 'notes';
export interface NoteBodyTabInput {
+ isUpload: boolean;
usesSegmentTranscript: boolean;
/** The transcript is still being made or failed, so its tab shows that status. */
transcriptPending: boolean;
- /** Uploaded audio stores its flat transcript as the note body; it has no notes of its own. */
+ /** An upload from before desktop stored its text as the transcript: its body is that text. */
contentIsTranscript: boolean;
hasEnhanced: boolean;
}
@@ -20,6 +21,7 @@ const NOTE_BODY_TAB_LABELS: Record<NoteBodyView, string> = {
* an upload's transcript have nowhere else to go, so they keep their tab.
*/
export function getNoteBodyTabs({
+ isUpload,
usesSegmentTranscript,
transcriptPending,
contentIsTranscript,
@@ -27,10 +29,9 @@ export function getNoteBodyTabs({
}: NoteBodyTabInput): NoteBodyView[] {
const tabs: NoteBodyView[] = [];
if (hasEnhanced) tabs.push('enhanced');
- const showTranscript =
- transcriptPending || (usesSegmentTranscript && (!hasEnhanced || contentIsTranscript));
+ const showTranscript = transcriptPending || (usesSegmentTranscript && (!hasEnhanced || isUpload));
if (showTranscript) tabs.push('transcript');
- // An upload's body only duplicates its transcript, so it shows only when there is no other.
+ // A legacy upload's body duplicates its transcript, so it shows only when there is no other.
if (!(contentIsTranscript && showTranscript)) tabs.push('notes');
return tabs;
}
diff --git a/openwhispr-mobile/src/screens/NoteEditorScreen.tsx b/openwhispr-mobile/src/screens/NoteEditorScreen.tsx
index b7383e24e..daba69398 100644
--- a/openwhispr-mobile/src/screens/NoteEditorScreen.tsx
+++ b/openwhispr-mobile/src/screens/NoteEditorScreen.tsx
@@ -455,8 +455,9 @@ export default function NoteEditorScreen({
const lastEnhancementInputHashRef = useRef('');
const usesSegmentTranscriptRef = useRef(usesSegmentTranscript);
usesSegmentTranscriptRef.current = usesSegmentTranscript;
- // Desktop stores an uploaded file's flat transcript as the note body.
- const contentIsTranscript = note?.noteType === 'upload';
+ // Desktop used to store an uploaded file's flat transcript as the note body. It now
+ // stores it as the transcript, leaving the body for the user's own notes.
+ const contentIsTranscript = note?.noteType === 'upload' && !hasTranscriptSegments;
// Notes typed beside a recording, never the transcript itself.
const hasTypedMeetingNotes = isAudioTranscript && !contentIsTranscript;
const activeSpeaker =
@@ -490,12 +491,13 @@ export default function NoteEditorScreen({
const hasEnhancedTab = hasEnhanced || enhancedEditing;
const bodyTabInput = useMemo(
() => ({
+ isUpload: note?.noteType === 'upload',
usesSegmentTranscript,
transcriptPending,
contentIsTranscript,
hasEnhanced: hasEnhancedTab,
}),
- [contentIsTranscript, hasEnhancedTab, transcriptPending, usesSegmentTranscript],
+ [contentIsTranscript, hasEnhancedTab, note?.noteType, transcriptPending, usesSegmentTranscript],
);
// A private note asks before any AI leaves this iPhone. Local Mode asks too for note chat; note
// formatting follows its own selection there.
diff --git a/openwhispr-mobile/src/screens/__tests__/NoteEditorScreen.calendarContext.test.tsx b/openwhispr-mobile/src/screens/__tests__/NoteEditorScreen.calendarContext.test.tsx
index 2c850c823..bb01b351f 100644
--- a/openwhispr-mobile/src/screens/__tests__/NoteEditorScreen.calendarContext.test.tsx
+++ b/openwhispr-mobile/src/screens/__tests__/NoteEditorScreen.calendarContext.test.tsx
@@ -1182,34 +1182,54 @@ describe('NoteEditorScreen body tabs', () => {
expect(getByText('Transcript failed')).toBeTruthy();
});
- it('never shows an uploaded file’s body, the flat copy of its transcript, as My notes', () => {
+ it('lets an upload with a transcript keep editable notes without changing its transcript', () => {
+ jest.useFakeTimers();
mockNote = note({
noteType: 'upload',
calendarEventId: null,
participants: null,
- content: 'Alice can take the first pass.',
+ content: '',
});
mockNotesState.notes = [mockNote];
- const { getByText, queryByTestId } = render(<NoteEditorScreen />);
+ const { getByText, getByTestId, queryByTestId } = render(<NoteEditorScreen />);
+ expect(getByText('Alice can take the first pass.')).toBeTruthy();
+ expect(queryByTestId('note-content-input')).toBeNull();
+
+ fireEvent.press(getByTestId('note-tab-notes'));
+ expect(getByTestId('note-content-input').props.value).toBe('');
+ expect(getByTestId('note-content-input').props.editable).toBe(true);
+ expect(getByTestId('note-content-input').props.placeholder).toBe('Add your own notes…');
+ fireEvent.changeText(getByTestId('note-content-input'), 'Send the revised checklist.');
+ act(() => {
+ jest.advanceTimersByTime(800);
+ });
+ expect(mockUpdateNote).toHaveBeenCalledWith(7, { content: 'Send the revised checklist.' });
+ expect(extractCorrections).not.toHaveBeenCalled();
+
+ fireEvent.press(getByTestId('note-tab-transcript'));
expect(getByText('Alice can take the first pass.')).toBeTruthy();
- expect(queryByTestId('note-tab-notes')).toBeNull();
expect(queryByTestId('note-content-input')).toBeNull();
});
- it('keeps an uploaded file’s transcript as a tab once notes are generated', () => {
+ it('keeps an upload’s transcript tab and typed notes beside generated notes', () => {
mockNote = note({
noteType: 'upload',
calendarEventId: null,
participants: null,
+ content: 'Send the revised checklist.',
enhancedContent: '## Summary',
});
mockNotesState.notes = [mockNote];
const { getByText, getByTestId, queryByTestId } = render(<NoteEditorScreen />);
+ expect(getByText('## Summary')).toBeTruthy();
expect(getByTestId('note-tab-enhanced')).toBeTruthy();
- expect(queryByTestId('note-tab-notes')).toBeNull();
+ expect(getByTestId('note-tab-transcript')).toBeTruthy();
- fireEvent.press(getByTestId('note-tab-transcript'));
+ fireEvent.press(getByTestId('note-tab-notes'));
+ expect(getByTestId('note-content-input').props.value).toBe('Send the revised checklist.');
+ expect(getByTestId('note-content-input').props.editable).toBe(true);
+ fireEvent.press(getByTestId('note-tab-transcript'));
expect(getByText('Alice can take the first pass.')).toBeTruthy();
expect(queryByTestId('note-content-input')).toBeNull();
});
diff --git a/preload.js b/preload.js
index 2370ffa82..8e32250fe 100644
--- a/preload.js
+++ b/preload.js
@@ -220,7 +220,7 @@ contextBridge.exposeInMainWorld("electronAPI", {
undoLearnedCorrections: (words) => ipcRenderer.invoke("undo-learned-corrections", words),
// Note functions
- saveNote: (title, content, noteType, sourceFile, audioDuration, folderId, spaceId) =>
+ saveNote: (title, content, noteType, sourceFile, audioDuration, folderId, spaceId, transcript) =>
ipcRenderer.invoke(
"db-save-note",
title,
@@ -229,7 +229,8 @@ contextBridge.exposeInMainWorld("electronAPI", {
sourceFile,
audioDuration,
folderId,
- spaceId
+ spaceId,
+ transcript
),
getNote: (id) => ipcRenderer.invoke("db-get-note", id),
getNotes: (noteType, limit, folderId, spaceId) =>
diff --git a/src/components/CommandSearch.tsx b/src/components/CommandSearch.tsx
index be534d606..e582a9265 100644
--- a/src/components/CommandSearch.tsx
+++ b/src/components/CommandSearch.tsx
@@ -25,6 +25,7 @@ import {
} from "./ui/dropdown-menu";
import type { NoteItem, FolderItem, SpaceItem, TranscriptionItem } from "../types/electron.js";
import { formatRelativeTime } from "../utils/dateFormatting";
+import { parseTranscriptSegments } from "../utils/parseTranscriptSegments";
import { defaultFolderDisplayName, folderMatchesQuery } from "./notes/shared";
interface ConversationResult {
@@ -666,7 +667,16 @@ function NoteRow({
t: (key: string, opts?: Record<string, unknown>) => string;
locale?: string;
}) {
- const preview = stripMarkdownPreview(note.content).slice(0, 90);
+ // An upload's transcription lives in its transcript, so one without notes previews that.
+ const preview = useMemo(() => {
+ const transcriptOnly = note.note_type === "upload" && !note.content.trim();
+ const text = transcriptOnly
+ ? parseTranscriptSegments(note.transcript || "")
+ .map((segment) => segment.text)
+ .join(" ")
+ : note.content;
+ return stripMarkdownPreview(text).slice(0, 90);
+ }, [note.content, note.note_type, note.transcript]);
const NoteIcon =
note.note_type === "meeting" ? Users : note.note_type === "upload" ? Upload : FileText;
return (
diff --git a/src/components/notes/NoteEditor.tsx b/src/components/notes/NoteEditor.tsx
index 8ee64accb..c468429d4 100644
--- a/src/components/notes/NoteEditor.tsx
+++ b/src/components/notes/NoteEditor.tsx
@@ -274,7 +274,15 @@ export default function NoteEditor({
}: NoteEditorProps) {
const { t } = useTranslation();
const locale = useUiLocale();
- const defaultViewMode: MeetingViewMode = enhancement ? "enhanced" : "raw";
+ // An upload's transcription lives in its transcript, so until the user writes
+ // notes it opens there rather than on an empty "Your notes".
+ const opensOnTranscript =
+ note.note_type === "upload" && !!note.transcript && !note.content.trim();
+ const defaultViewMode: MeetingViewMode = enhancement
+ ? "enhanced"
+ : opensOnTranscript
+ ? "transcript"
+ : "raw";
const [selectedViewMode, setSelectedViewMode] = useState<MeetingViewMode>(defaultViewMode);
// Stored as chosen, clamped on read: AI Summary is the only tab that can stop
// rendering, and a tab that no longer renders can never be the current one.
diff --git a/src/components/notes/PersonalNotesView.tsx b/src/components/notes/PersonalNotesView.tsx
index 47bec7beb..8d8eb4475 100644
--- a/src/components/notes/PersonalNotesView.tsx
+++ b/src/components/notes/PersonalNotesView.tsx
@@ -23,7 +23,11 @@ import {
} from "../../stores/settingsStore";
import { cn } from "../lib/utils";
import logger from "../../utils/logger";
-import { parseSeedSegments, parseTranscriptSegments } from "../../utils/parseTranscriptSegments";
+import {
+ parseSeedSegments,
+ parseTranscriptSegments,
+ speakerlessTranscriptText,
+} from "../../utils/parseTranscriptSegments";
import { isExplicitSpeakerCount, resolveExpectedSpeakerCount } from "../../utils/participants";
import {
buildLlmTranscript,
@@ -751,10 +755,14 @@ export default function PersonalNotesView({
const noteContent = editorNote.content;
const hasNotes = !!noteContent.trim();
if (!hasNotes && !rawTranscript) return;
- // An upload whose provider returned no segments saved its transcript as the
- // note's text and left the transcript empty (#2666).
+ // Uploads made before their text moved into the transcript keep it as the
+ // note's text with no transcript; newer ones whose provider named no
+ // speakers store it as segments without any. Either way nothing shows who
+ // speaks, so it runs as a plain transcript (#2666).
const transcriptInContent = !rawTranscript && editorNote.note_type === "upload";
- const transcriptText = transcriptInContent ? noteContent : rawTranscript;
+ const segments = parseTranscriptSegments(rawTranscript);
+ const speakerlessText = speakerlessTranscriptText(segments);
+ const transcriptText = transcriptInContent ? noteContent : (speakerlessText ?? rawTranscript);
let formattedTranscript = "";
let meetingContext = "";
@@ -767,8 +775,7 @@ export default function PersonalNotesView({
selfEmail: user?.email?.trim() || null,
participants: parseNoteParticipants(editorNote.participants),
};
- const segments = parseTranscriptSegments(rawTranscript);
- if (segments.length > 0) {
+ if (segments.length > 0 && speakerlessText === null) {
isMeetingNote = true;
const mappingRows =
(await window.electronAPI?.getSpeakerMappings?.(editorNote.id).catch(() => [])) || [];
diff --git a/src/helpers/database.js b/src/helpers/database.js
index 2b5c980a9..38ca4e42c 100644
--- a/src/helpers/database.js
+++ b/src/helpers/database.js
@@ -3205,7 +3205,8 @@ class DatabaseManager {
sourceFile = null,
audioDuration = null,
folderId = null,
- spaceId = null
+ spaceId = null,
+ transcript = null
) {
try {
if (!this.db) {
@@ -3232,7 +3233,7 @@ class DatabaseManager {
const clientNoteId = randomUUID();
const accountId = this._accountIdForSpace(spaceId);
const stmt = this.db.prepare(
- "INSERT INTO notes (title, content, content_sync_operation, note_type, source_file, audio_duration_seconds, folder_id, space_id, client_note_id, account_id) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
+ "INSERT INTO notes (title, content, content_sync_operation, note_type, source_file, audio_duration_seconds, folder_id, space_id, client_note_id, account_id, transcript) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
);
const result = stmt.run(
title,
@@ -3244,7 +3245,8 @@ class DatabaseManager {
folderId,
spaceId,
clientNoteId,
- accountId
+ accountId,
+ transcript
);
const fetchStmt = this.db.prepare("SELECT * FROM notes WHERE id = ?");
diff --git a/src/helpers/ipcHandlers.js b/src/helpers/ipcHandlers.js
index 76a965f80..f402d42f7 100644
--- a/src/helpers/ipcHandlers.js
+++ b/src/helpers/ipcHandlers.js
@@ -2284,7 +2284,17 @@ class IPCHandlers {
ipcMain.handle(
"db-save-note",
- async (event, title, content, noteType, sourceFile, audioDuration, folderId, spaceId) => {
+ async (
+ event,
+ title,
+ content,
+ noteType,
+ sourceFile,
+ audioDuration,
+ folderId,
+ spaceId,
+ transcript
+ ) => {
const result = this.databaseManager.saveNote(
title,
content,
@@ -2292,7 +2302,8 @@ class IPCHandlers {
sourceFile,
audioDuration,
folderId,
- spaceId
+ spaceId,
+ transcript
);
if (result?.success && result?.note) {
setImmediate(() => broadcastToWindows("note-added", result.note));
@@ -4208,7 +4219,7 @@ class IPCHandlers {
end: typeof s.end === "number" && isFinite(s.end) ? s.end : 0,
}));
const merged = mergeSpeakersWithText(sanitizedSegments, text, duration);
- return { success: true, text: formatSpeakerTranscript(merged) };
+ return { success: true, text: formatSpeakerTranscript(merged), segments: merged };
} catch (error) {
debugLogger.error("Speaker merge error", { error: error.message });
return { success: false, error: error.message };
@@ -10995,6 +11006,12 @@ class IPCHandlers {
debugLogger.warn("BYOK diarization requested but provider returned no speaker data");
}
const segments = timestamps ? mapVerboseSegments(data.data) : null;
+ if (timestamps && !segments) {
+ debugLogger.debug("BYOK transcription returned no usable segment timing", {
+ provider: route.provider,
+ hasSegments: Array.isArray(data.data?.segments),
+ });
+ }
return { success: true, text: data.data.text, ...(segments ? { segments } : {}) };
} catch (rawError) {
const error = providerErrorContext
diff --git a/src/helpers/transcriptFormatter.js b/src/helpers/transcriptFormatter.js
index 76bc66bf4..93b6a0082 100644
--- a/src/helpers/transcriptFormatter.js
+++ b/src/helpers/transcriptFormatter.js
@@ -87,6 +87,19 @@ function mergeSegments(segments) {
return merged;
}
+// An upload whose provider returned no segments stores its text as segments
+// with no timing, speaker or source: a "00:00:00" or "Unknown speaker" heading
+// on them would be invented. parseTranscriptSegments.ts's
+// speakerlessTranscriptText makes the matching call for note actions.
+function isUntimed(segments) {
+ return (
+ segments.length > 0 &&
+ segments.every(
+ (seg) => seg.timestamp == null && !seg.speaker && !seg.speakerName && !seg.source
+ )
+ );
+}
+
function formatTimestamp(seconds) {
const s = Math.floor(seconds);
const h = Math.floor(s / 3600);
@@ -130,8 +143,11 @@ function formatTxt(note, segments, speakerMappings) {
if (participants.length)
lines.push(`${i18nMain.t("notes.editor.participants")}: ${participants.join(", ")}`);
lines.push("", "──────────────────────────────────", "");
+ const untimed = isUntimed(segments);
for (const seg of merged) {
- lines.push(`[${formatTimestamp(seg.timestamp)}] ${resolveSpeaker(seg, speakerMappings)}:`);
+ if (!untimed) {
+ lines.push(`[${formatTimestamp(seg.timestamp)}] ${resolveSpeaker(seg, speakerMappings)}:`);
+ }
lines.push(seg.text);
lines.push("");
}
@@ -144,10 +160,14 @@ function formatSrt(segments, speakerMappings, note = {}) {
// with "Unknown Speaker:" would defeat the subtitle use case. Meeting
// segments always serialize a source, so they keep their labels.
const hasSpeakerIdentity = merged.some((seg) => seg.speaker || seg.speakerName || seg.source);
+ // Untimed text has one cue at 0; it spans the recording when its length is known.
+ const lastEnd =
+ isUntimed(segments) && note.audio_duration_seconds > 0 ? note.audio_duration_seconds : null;
const entries = [];
for (let i = 0; i < merged.length; i++) {
const seg = merged[i];
- const nextTs = i + 1 < merged.length ? merged[i + 1].timestamp : seg.endTimestamp + 3;
+ const nextTs =
+ i + 1 < merged.length ? merged[i + 1].timestamp : (lastEnd ?? seg.endTimestamp + 3);
entries.push(`${i + 1}`);
entries.push(`${formatSrtTimestamp(seg.timestamp)} --> ${formatSrtTimestamp(nextTs)}`);
entries.push(
@@ -162,8 +182,11 @@ function formatJson(note, segments, speakerMappings) {
const merged = mergeSegments(normalizeSegmentTimestamps(segments, note));
const { title, dateStr } = extractMetadata(note);
+ const untimed = isUntimed(segments);
const speakersSet = new Set();
- for (const seg of merged) speakersSet.add(resolveSpeaker(seg, speakerMappings));
+ if (!untimed) {
+ for (const seg of merged) speakersSet.add(resolveSpeaker(seg, speakerMappings));
+ }
const lastSeg = merged[merged.length - 1];
return JSON.stringify(
@@ -171,13 +194,18 @@ function formatJson(note, segments, speakerMappings) {
metadata: {
title,
date: dateStr,
- duration_seconds: lastSeg ? Math.round(lastSeg.endTimestamp) : 0,
+ duration_seconds:
+ untimed && note.audio_duration_seconds > 0
+ ? Math.round(note.audio_duration_seconds)
+ : lastSeg
+ ? Math.round(lastSeg.endTimestamp)
+ : 0,
speaker_count: speakersSet.size,
segment_count: merged.length,
},
speakers: [...speakersSet],
segments: merged.map((seg) => ({
- speaker: resolveSpeaker(seg, speakerMappings),
+ speaker: untimed ? null : resolveSpeaker(seg, speakerMappings),
timestamp: seg.timestamp,
text: seg.text,
})),
@@ -195,8 +223,13 @@ function formatMd(note, segments, speakerMappings) {
if (participants.length)
lines.push(`**${i18nMain.t("notes.editor.participants")}:** ${participants.join(", ")}`);
lines.push("", "---", "");
+ const untimed = isUntimed(segments);
for (const seg of merged) {
- lines.push(`**${resolveSpeaker(seg, speakerMappings)}** \`${formatTimestamp(seg.timestamp)}\``);
+ if (!untimed) {
+ lines.push(
+ `**${resolveSpeaker(seg, speakerMappings)}** \`${formatTimestamp(seg.timestamp)}\``
+ );
+ }
lines.push(`${seg.text}`, "");
}
return lines.join("\n");
diff --git a/src/helpers/uploadTimestamps.js b/src/helpers/uploadTimestamps.js
index 8ae7e0841..e01a48921 100644
--- a/src/helpers/uploadTimestamps.js
+++ b/src/helpers/uploadTimestamps.js
@@ -19,7 +19,9 @@ function timestampRequestFields(provider, model) {
// verbose_json (OpenAI/Groq) and Mistral's segment granularity both carry
// `segments: [{ text, start, end }]`. Null when the response has nothing
-// usable, so callers fall back to the plain transcript.
+// usable, or when the kept segments don't hold every word of the response's
+// text (whitespace aside, so CJK joins match), so callers fall back to the
+// plain transcript rather than store a timed one with words missing.
function mapVerboseSegments(responseData) {
const segments = responseData?.segments;
if (!Array.isArray(segments)) return null;
@@ -37,7 +39,14 @@ function mapVerboseSegments(responseData) {
end,
});
}
- return mapped.length ? mapped : null;
+ if (!mapped.length) return null;
+ if (typeof responseData.text === "string") {
+ const compact = (value) => value.replace(/\s+/g, "");
+ if (compact(mapped.map((seg) => seg.text).join("")) !== compact(responseData.text)) {
+ return null;
+ }
+ }
+ return mapped;
}
module.exports = { timestampRequestFields, mapVerboseSegments };
diff --git a/src/services/fileTranscription.ts b/src/services/fileTranscription.ts
index 5de41f085..4bd6ccfd0 100644
--- a/src/services/fileTranscription.ts
+++ b/src/services/fileTranscription.ts
@@ -28,7 +28,8 @@ export interface FileTranscriptionResult {
// audio_duration_seconds. Only transcribeFileWithSpeakers sets it.
durationSeconds?: number | null;
// Segment-level timing from BYOK providers that support it (opts.timestamps
- // or BYOK diarization). Absent whenever the provider returned text only.
+ // or BYOK diarization), with local diarization's speakers when that ran, or
+ // the merged speaker turns when the provider returned text only.
segments?: Array<{ text: string; start: number; end: number; speaker?: string }>;
}
@@ -243,6 +244,27 @@ export async function resolveDiarizationSettings({
};
}
+type SpeakerSpan = { start: number; end: number; speaker: string };
+
+function assignSpeakers(
+ segments: NonNullable<FileTranscriptionResult["segments"]>,
+ speakerSpans: SpeakerSpan[]
+): NonNullable<FileTranscriptionResult["segments"]> {
+ return segments.map((segment) => {
+ let best = speakerSpans[0];
+ let bestScore = Number.NEGATIVE_INFINITY;
+ for (const span of speakerSpans) {
+ // Positive: seconds of overlap. Negative: the gap to a span that doesn't overlap.
+ const score = Math.min(segment.end, span.end) - Math.max(segment.start, span.start);
+ if (score > bestScore) {
+ best = span;
+ bestScore = score;
+ }
+ }
+ return { ...segment, speaker: best.speaker };
+ });
+}
+
// Transcribe and diarize in parallel, then merge speaker labels into the text.
// Shared by the single-file flow and the batch queue. `durationSeconds` (when the
// source knows it, e.g. URL downloads) beats inferring duration from segments.
@@ -283,9 +305,20 @@ export async function transcribeFileWithSpeakers(
const merged = await window.electronAPI.mergeSpeakerText?.(
diar.segments,
result.text,
- durationSeconds || 0
+ measuredDuration || 0
);
- if (merged?.success && merged.text) return { ...result, text: merged.text };
+ if (merged?.success && merged.text) {
+ // The transcript keeps who spoke when: the provider's timed segments get
+ // the speaker they overlap most, and without them the merged turns
+ // (placed by their share of the text) stand in.
+ return {
+ ...result,
+ text: merged.text,
+ segments: result.segments
+ ? assignSpeakers(result.segments, diar.segments)
+ : merged.segments,
+ };
+ }
} catch {
// Merge failure falls back to the plain transcript.
}
diff --git a/src/services/uploadNotes.ts b/src/services/uploadNotes.ts
index d21225c55..560cd8ed3 100644
--- a/src/services/uploadNotes.ts
+++ b/src/services/uploadNotes.ts
@@ -16,21 +16,31 @@ export interface UploadSegment {
// mixed-base transcript defeats the export rebase (normalizeSegmentTimestamps
// keys off min timestamp > 1e9), rendering garbage cue times. The rebase
// subtracts the minimum on export, so pure-upload output is unchanged.
-// Undefined when there is nothing usable, so plain-text uploads stay exactly
-// as they were.
+// Speakers are stored as the meeting path's ids (speaker_0, … in order of first
+// appearance), never as names: the UI translates "Speaker N" from them, renaming
+// keys its mappings on them, mobile reads them, and a recording later made into
+// the note numbers its own speakers after them.
+// Undefined when there is nothing usable. The transcript is the note's only
+// copy of the text, so a segment with words but no start (no producer emits one
+// today) drops the timing altogether and the caller stores the text untimed.
export function buildUploadTranscript(
segments?: UploadSegment[] | null,
anchorMs: number = Date.now()
): string | undefined {
- if (!segments?.length) return undefined;
- const stored = segments
- .filter((seg) => seg.text?.trim() && Number.isFinite(seg.start))
- .map((seg) => ({
+ const spoken = segments?.filter((seg) => seg.text?.trim()) ?? [];
+ if (!spoken.length || spoken.some((seg) => !Number.isFinite(seg.start))) return undefined;
+ const speakerIds = new Map<string, string>();
+ const stored = spoken.map((seg) => {
+ if (seg.speaker && !speakerIds.has(seg.speaker)) {
+ speakerIds.set(seg.speaker, `speaker_${speakerIds.size}`);
+ }
+ return {
text: seg.text.trim(),
timestamp: anchorMs + seg.start * 1000,
- ...(seg.speaker ? { speakerName: seg.speaker } : {}),
- }));
- return stored.length ? JSON.stringify(stored) : undefined;
+ ...(seg.speaker ? { speaker: speakerIds.get(seg.speaker) } : {}),
+ };
+ });
+ return JSON.stringify(stored);
}
// What an upload persists onto the note row about the diarization run,
@@ -47,20 +57,13 @@ export function buildUploadNoteMetadata(
segments?: UploadSegment[] | null,
anchorMs?: number
) {
- const transcript = buildUploadTranscript(segments, anchorMs);
- const noteUpdates: Record<string, unknown> | null =
- diarization.enabled || transcript
- ? {
- ...(diarization.enabled ? { diarization_enabled: 1 } : {}),
- ...(transcript ? { transcript } : {}),
- }
- : null;
return {
+ transcript: buildUploadTranscript(segments, anchorMs),
+ noteUpdates: diarization.enabled ? { diarization_enabled: 1 } : null,
audioDurationSeconds:
typeof durationSeconds === "number" && Number.isFinite(durationSeconds) && durationSeconds > 0
? durationSeconds
: null,
- noteUpdates,
};
}
@@ -75,10 +78,12 @@ interface SaveUploadNoteParams {
}
// The one save path for upload and URL-ingest notes, shared by the single-file
-// flow and the batch queue: the duration goes in the insert (updateNote does
-// not whitelist audio_duration_seconds), and the diarization columns and the
-// timestamped transcript follow through updateNote — the same route the
-// meeting path writes them through.
+// flow and the batch queue. The transcription goes in the insert as the note's
+// transcript, timed when the provider gave segments and untimed otherwise, so
+// "Your notes" stays the user's own and the note-added event and file mirror
+// see the text even if the later metadata write fails. The duration goes in the
+// insert too (updateNote does not whitelist audio_duration_seconds); the
+// diarization column follows through updateNote, as on the meeting path.
export async function saveUploadNote({
title,
text,
@@ -88,18 +93,20 @@ export async function saveUploadNote({
durationSeconds,
segments,
}: SaveUploadNoteParams) {
- const { audioDurationSeconds, noteUpdates } = buildUploadNoteMetadata(
+ const { transcript, audioDurationSeconds, noteUpdates } = buildUploadNoteMetadata(
diarization,
durationSeconds,
segments
);
const res = await window.electronAPI.saveNote(
title,
- text,
+ "",
"upload",
sourceName,
audioDurationSeconds,
- folderId
+ folderId,
+ null,
+ transcript ?? JSON.stringify([{ text }])
);
if (res.success && res.note && noteUpdates) {
// Best-effort: a failed metadata write must not error a saved note.
diff --git a/src/types/electron.ts b/src/types/electron.ts
index 33639c930..09a446786 100644
--- a/src/types/electron.ts
+++ b/src/types/electron.ts
@@ -1707,7 +1707,8 @@ declare global {
sourceFile?: string | null,
audioDuration?: number | null,
folderId?: number | null,
- spaceId?: number | null
+ spaceId?: number | null,
+ transcript?: string | null
) => Promise<{ success: boolean; note?: NoteItem }>;
getNote: (id: number) => Promise<NoteItem | null>;
getNotes: (
@@ -3371,7 +3372,12 @@ declare global {
segments: Array<{ start: number; end: number; speaker: string }>,
text: string,
duration: number
- ) => Promise<{ success: boolean; text?: string; error?: string }>;
+ ) => Promise<{
+ success: boolean;
+ text?: string;
+ segments?: Array<{ text: string; start: number; end: number; speaker: string }>;
+ error?: string;
+ }>;
diarizeAudioFile?: (
filePath: string,
options?: { numSpeakers?: number; threshold?: number; requestId?: string }
diff --git a/src/utils/parseTranscriptSegments.ts b/src/utils/parseTranscriptSegments.ts
index 53c13a561..a55b712ca 100644
--- a/src/utils/parseTranscriptSegments.ts
+++ b/src/utils/parseTranscriptSegments.ts
@@ -39,6 +39,21 @@ export function parseTranscriptSegments(raw: string): TranscriptSegment[] {
}
}
+/**
+ * The text of a transcript none of whose segments says who speaks (no speaker,
+ * speaker name or source), joined with blank lines; null otherwise. Uploads
+ * whose provider returned no speakers store their text this way, so note
+ * actions run it as a plain transcript (#2666), not a meeting where every line
+ * would be "Them". transcriptFormatter.js's `isSpeakerless` makes the same call
+ * for exports.
+ */
+export function speakerlessTranscriptText(segments: TranscriptSegment[]): string | null {
+ const speakerless =
+ segments.length > 0 &&
+ segments.every((segment) => !segment.source && !segment.speaker && !segment.speakerName);
+ return speakerless ? segments.map((segment) => segment.text).join("\n\n") : null;
+}
+
const isJsonArray = (raw: string): boolean => {
try {
return Array.isArray(JSON.parse(raw));
diff --git a/test/components/noteEditorSummaryDeletion.test.js b/test/components/noteEditorSummaryDeletion.test.js
index 5dbef9d88..2e717e878 100644
--- a/test/components/noteEditorSummaryDeletion.test.js
+++ b/test/components/noteEditorSummaryDeletion.test.js
@@ -286,6 +286,57 @@ test("deleting the AI summary moves the selection to Your notes", async (t) => {
await unmount();
});
+test("an uploaded transcript opens in Transcript and leaves Your notes available for writing", async (t) => {
+ const { createDb } = require("../helpers/harness/db");
+ const db = createDb(t);
+ assert.ok(db);
+ let cleanup = () => {};
+ t.after(() => cleanup());
+ const { render, click, latest, unmount, vite } = await loadNoteEditor(t);
+ cleanup = unmount;
+ Object.assign(globalThis.window.electronAPI, {
+ saveNote: async (...args) => db.saveNote(...args),
+ updateNote: async (...args) => db.updateNote(...args),
+ });
+ const { saveUploadNote } = await vite.ssrLoadModule("/services/uploadNotes.ts");
+ const text = "Uploaded words.\nAnother paragraph.";
+ const saved = await saveUploadNote({
+ title: "Uploaded meeting",
+ text,
+ sourceName: "public-fixture.wav",
+ folderId: null,
+ diarization: { enabled: false, localModelsReady: false, numSpeakers: null },
+ });
+ const note = db.getNote(saved.note.id);
+ await render(undefined, { note });
+ // No click: an upload without notes opens on its transcript.
+ let transcript = null;
+ walk(latest(), (node) => {
+ if (node.props.segments) transcript = node.props.segments;
+ });
+ assert.deepEqual(
+ transcript?.map((segment) => segment.text),
+ [text]
+ );
+ await click("raw");
+ assert.ok(collectEditorValues(latest()).includes(""), "Your notes starts empty");
+ db.updateNote(note.id, { content: "My own reminder" });
+ await render(undefined, { note: db.getNote(note.id) });
+ await click("raw");
+ assert.ok(collectEditorValues(latest()).includes("My own reminder"));
+ await click("transcript");
+ transcript = null;
+ walk(latest(), (node) => {
+ if (node.props.segments) transcript = node.props.segments;
+ });
+ assert.deepEqual(
+ transcript.map((segment) => segment.text),
+ [text]
+ );
+ await unmount();
+ cleanup = () => {};
+});
+
test("the highlight slides onto Your notes when the summary is deleted", async (t) => {
const { render, click, latest, unmount } = await loadNoteEditor(t);
diff --git a/test/components/personalNotesUndoDiscard.test.js b/test/components/personalNotesUndoDiscard.test.js
index bc5e891d2..36423e538 100644
--- a/test/components/personalNotesUndoDiscard.test.js
+++ b/test/components/personalNotesUndoDiscard.test.js
@@ -351,8 +351,46 @@ test("a note without a transcript keeps the personal-note run", async (t) => {
assert.deepEqual(options.knownPeople, []);
});
-// An upload whose provider returned no segments saves its transcript as the
-// note's text and leaves the transcript empty, so that text is the transcript.
+// Uploads whose provider named no speakers store segments with no speaker or
+// source (and with no timing when it returned no segments): nothing shows who
+// speaks, so they must not run as a meeting with every line given to "Them" (#2666).
+test("an upload stored as untimed segments runs as a plain transcript beside the user's notes", async (t) => {
+ const text = "Alice, can you fix the build by Friday? Sure.";
+ const { input, options } = await runSummaryOn(t, {
+ note_type: "upload",
+ transcript: JSON.stringify([{ text }]),
+ participants: PARTICIPANTS,
+ });
+ assert.equal(options.hasPlainTranscript, true);
+ assert.equal(options.isMeetingNote, false);
+ assert.equal(input, `${NOTE.content}\n\n## Meeting Transcript\n${text}`);
+ assert.deepEqual(options.material, { notes: NOTE.content, meetingContext: "", transcript: text });
+ assert.deepEqual(options.knownPeople, USER_AND_INVITEE);
+});
+
+test("an upload with timed segments but no speakers runs as a plain transcript", async (t) => {
+ const transcript = JSON.stringify([
+ { text: "First part.", timestamp: 1_700_000_000_000 },
+ { text: "Second part.", timestamp: 1_700_000_004_000 },
+ ]);
+ const { input, options } = await runSummaryOn(t, { note_type: "upload", transcript });
+ assert.equal(options.hasPlainTranscript, true);
+ assert.equal(options.isMeetingNote, false);
+ assert.ok(input.endsWith("## Meeting Transcript\nFirst part.\n\nSecond part."));
+});
+
+test("an upload with timed speaker segments still runs as a meeting", async (t) => {
+ const transcript = JSON.stringify([
+ { text: "Ship on Friday.", timestamp: 1_700_000_000_000, speaker: "speaker_0" },
+ ]);
+ const { input, options } = await runSummaryOn(t, { note_type: "upload", transcript });
+ assert.equal(options.isMeetingNote, true);
+ assert.ok(!options.hasPlainTranscript);
+ assert.ok(input.includes("Speaker 1: Ship on Friday."));
+});
+
+// Uploads made before the transcript held their text saved it as the note's
+// text and left the transcript empty, so that text is the transcript.
test("an upload whose transcript is the note's text runs as a plain transcript", async (t) => {
const content = "Alice, can you fix the build by Friday? Sure.";
const { input, options } = await runSummaryOn(t, {
diff --git a/test/helpers/transcriptFormatter.test.js b/test/helpers/transcriptFormatter.test.js
index 9246b1c16..709dad842 100644
--- a/test/helpers/transcriptFormatter.test.js
+++ b/test/helpers/transcriptFormatter.test.js
@@ -303,3 +303,31 @@ test("SQLite UTC note timestamps are parsed as UTC before local formatting", ()
assert.equal(parsed.toISOString(), "2026-08-04T13:02:00.000Z");
});
+
+// An upload whose provider returned no segments stores its text untimed; a
+// "[00:00:00] Unknown speaker" heading or a 3-second cue would be invented.
+test("untimed upload text exports without invented times or speakers", () => {
+ const note = {
+ title: "Lecture",
+ created_at: "2026-01-01T00:00:00Z",
+ audio_duration_seconds: 125.4,
+ };
+ const segments = [{ text: "The whole lecture." }];
+
+ const txt = formatTxt(note, segments, {});
+ assert.ok(txt.endsWith("──────────────────────────────────\n\nThe whole lecture.\n"));
+ assert.doesNotMatch(txt, /\[00:00:00\]/);
+
+ const md = formatMd(note, segments, {});
+ assert.ok(md.endsWith("---\n\nThe whole lecture.\n"));
+ assert.doesNotMatch(md, /00:00:00|Unknown/);
+
+ assert.equal(
+ formatSrt(segments, {}, note),
+ "1\n00:00:00,000 --> 00:02:05,400\nThe whole lecture.\n"
+ );
+ const json = JSON.parse(formatJson(note, segments, {}));
+ assert.equal(json.metadata.duration_seconds, 125);
+ assert.deepEqual(json.speakers, []);
+ assert.equal(json.segments[0].speaker, null);
+});
diff --git a/test/helpers/uploadTimestamps.test.js b/test/helpers/uploadTimestamps.test.js
index e88b1eeb8..16e6e10fe 100644
--- a/test/helpers/uploadTimestamps.test.js
+++ b/test/helpers/uploadTimestamps.test.js
@@ -81,6 +81,34 @@ test("verbose segments clamp negative start timestamps and inverted end timestam
]);
});
+// The segments become the upload note's only copy of the text, so they must
+// hold every word of it; a CJK response's text has no spaces between them.
+test("segments must cover the response text, whitespace aside", () => {
+ assert.deepEqual(
+ mapVerboseSegments({
+ text: "你好。今天开会。",
+ segments: [
+ { start: 0, end: 1, text: "你好。" },
+ { start: 1, end: 2, text: "今天开会。" },
+ ],
+ }),
+ [
+ { text: "你好。", start: 0, end: 1 },
+ { text: "今天开会。", start: 1, end: 2 },
+ ]
+ );
+ assert.equal(
+ mapVerboseSegments({
+ text: "First sentence. Second sentence.",
+ segments: [
+ { start: 0, end: 1, text: " First sentence." },
+ { start: NaN, end: 2, text: " Second sentence." },
+ ],
+ }),
+ null
+ );
+});
+
test("responses without usable segments map to null", () => {
assert.equal(mapVerboseSegments({ text: "plain json response" }), null);
assert.equal(mapVerboseSegments({ segments: "not-an-array" }), null);
diff --git a/test/services/fileTranscriptionTimestamps.test.js b/test/services/fileTranscriptionTimestamps.test.js
index f46b3470f..d773f4436 100644
--- a/test/services/fileTranscriptionTimestamps.test.js
+++ b/test/services/fileTranscriptionTimestamps.test.js
@@ -62,3 +62,65 @@ test("timestamps opt-in reaches the BYOK IPC call and segments flow back", async
assert.deepEqual(withSpeakers.segments, segments);
assert.equal(withSpeakers.durationSeconds, 120);
});
+
+// Local diarization over a provider that returned timed segments: each keeps
+// its own timing and takes the diarizer speaker it overlaps most, instead of
+// the merge's text-proportional turns.
+test("local diarization labels the provider's timed segments by overlap", async (t) => {
+ const { window } = installBrowserGlobals(t);
+ const vite = await createRendererServer(t, {
+ cachePrefix: "openwhispr-file-timestamps-diarize-test-",
+ mockModules: {
+ "/lib/auth": "export const withSessionRefresh = (fn) => fn();",
+ },
+ });
+ const { transcribeFileWithSpeakers } = await vite.ssrLoadModule("/services/fileTranscription.ts");
+
+ window.electronAPI.transcribeAudioFileByok = async () => ({
+ success: true,
+ text: "Hello. Reply. Gap.",
+ segments: [
+ { text: "Hello.", start: 0, end: 4 },
+ { text: "Reply.", start: 5, end: 9 },
+ { text: "Gap.", start: 20, end: 21 },
+ ],
+ });
+ let mergeDuration = null;
+ window.electronAPI.diarizeAudioFile = async () => ({
+ success: true,
+ durationSeconds: 30,
+ segments: [
+ { start: 0, end: 4.5, speaker: "speaker_1" },
+ { start: 4.5, end: 10, speaker: "speaker_0" },
+ { start: 18, end: 19, speaker: "speaker_1" },
+ ],
+ });
+ window.electronAPI.mergeSpeakerText = async (_segments, _text, duration) => {
+ mergeDuration = duration;
+ return {
+ success: true,
+ text: "[Speaker 1] 0:00 - 0:30\nHello. Reply. Gap.",
+ segments: [{ speaker: "speaker_1", text: "Hello. Reply. Gap.", start: 0, end: 30 }],
+ };
+ };
+
+ // Groq returns timed segments and has no diarization of its own.
+ const result = await transcribeFileWithSpeakers(
+ "/tmp/audio.webm",
+ {
+ ...byokConfig(),
+ cloudTranscriptionProvider: "groq",
+ cloudTranscriptionModel: "whisper-large-v3",
+ },
+ { enabled: true, localModelsReady: true, numSpeakers: null },
+ null,
+ { timestamps: true }
+ );
+
+ assert.equal(mergeDuration, 30, "the merge gets the diarizer's measured duration");
+ assert.deepEqual(result.segments, [
+ { text: "Hello.", start: 0, end: 4, speaker: "speaker_1" },
+ { text: "Reply.", start: 5, end: 9, speaker: "speaker_0" },
+ { text: "Gap.", start: 20, end: 21, speaker: "speaker_1" },
+ ]);
+});
diff --git a/test/services/uploadDiarizationPersistence.test.js b/test/services/uploadDiarizationPersistence.test.js
index 21dbccb8e..490e9f481 100644
--- a/test/services/uploadDiarizationPersistence.test.js
+++ b/test/services/uploadDiarizationPersistence.test.js
@@ -58,7 +58,14 @@ function installUploadElectronAPI(window, { diarizedDurationSeconds }) {
durationSeconds: diarizedDurationSeconds,
};
},
- mergeSpeakerText: async () => ({ success: true, text: "[Speaker 1] hello [Speaker 2] hi" }),
+ mergeSpeakerText: async () => ({
+ success: true,
+ text: "[Speaker 1] 0:00 - 0:30\nhello\n\n[Speaker 2] 0:30 - 1:00\nhi",
+ segments: [
+ { speaker: "Speaker 1", text: "hello", start: 0, end: 30 },
+ { speaker: "Speaker 2", text: "hi", start: 30, end: 60 },
+ ],
+ }),
saveNote: async (...args) => {
calls.saveNote.push(args);
return { success: true, note: { id: 7 } };
@@ -93,7 +100,19 @@ test("a diarized file upload persists the diarization metadata", async (t) => {
assert.equal(calls.saveNote.length, 1);
const [, content, noteType, sourceFile, audioDuration] = calls.saveNote[0];
- assert.equal(content, "[Speaker 1] hello [Speaker 2] hi");
+ // The transcription is the note's transcript, split at the merged speaker
+ // turns (local whisper returns no segments of its own), so "Your notes"
+ // stays empty for the user's own.
+ assert.equal(content, "");
+ const transcript = JSON.parse(calls.saveNote[0][7]);
+ assert.deepEqual(
+ transcript.map(({ text, speaker }) => ({ text, speaker })),
+ [
+ { text: "hello", speaker: "speaker_0" },
+ { text: "hi", speaker: "speaker_1" },
+ ]
+ );
+ assert.equal(transcript[1].timestamp - transcript[0].timestamp, 30_000);
assert.equal(noteType, "upload");
assert.equal(sourceFile, "board-meeting.m4a");
assert.equal(audioDuration, 4359.87);
diff --git a/test/services/uploadNotes.test.js b/test/services/uploadNotes.test.js
index 11279df1a..0cc23d40a 100644
--- a/test/services/uploadNotes.test.js
+++ b/test/services/uploadNotes.test.js
@@ -76,7 +76,7 @@ test("provider segments serialize into the meeting-path transcript shape", async
// relative seconds with Date.now() values.
assert.deepEqual(JSON.parse(transcript), [
{ text: "Hello world.", timestamp: anchorMs },
- { text: "Named part.", timestamp: anchorMs + 4200, speakerName: "Speaker 1" },
+ { text: "Named part.", timestamp: anchorMs + 4200, speaker: "speaker_0" },
]);
});
@@ -139,37 +139,86 @@ test("segments persist the transcript even when diarization is off", async () =>
const { buildUploadNoteMetadata } = await load();
const anchorMs = 1_700_000_000_000;
- const { noteUpdates } = buildUploadNoteMetadata(
+ const { transcript, noteUpdates } = buildUploadNoteMetadata(
{ enabled: false, localModelsReady: false, numSpeakers: null },
60,
[{ text: "Hello.", start: 0, end: 2 }],
anchorMs
);
+ assert.equal(transcript, JSON.stringify([{ text: "Hello.", timestamp: anchorMs }]));
// The transcript must not drag the diarization columns along: a null
// diarization_enabled still defers to the global speaker setting.
- assert.deepEqual(noteUpdates, {
- transcript: JSON.stringify([{ text: "Hello.", timestamp: anchorMs }]),
- });
+ assert.equal(noteUpdates, null);
});
-test("segments and diarization metadata share one noteUpdates write", async () => {
+test("diarization metadata follows the insert separately from the transcript", async () => {
const { buildUploadNoteMetadata } = await load();
const anchorMs = 1_700_000_000_000;
- const { noteUpdates } = buildUploadNoteMetadata(
+ const { transcript, noteUpdates } = buildUploadNoteMetadata(
{ enabled: true, localModelsReady: true, numSpeakers: 2 },
60,
[{ text: "Hi.", start: 1, speaker: "Speaker 1" }],
anchorMs
);
- assert.deepEqual(noteUpdates, {
- diarization_enabled: 1,
- transcript: JSON.stringify([
- { text: "Hi.", timestamp: anchorMs + 1000, speakerName: "Speaker 1" },
+ assert.equal(
+ transcript,
+ JSON.stringify([{ text: "Hi.", timestamp: anchorMs + 1000, speaker: "speaker_0" }])
+ );
+ assert.deepEqual(noteUpdates, { diarization_enabled: 1 });
+});
+
+// The transcript is the note's only copy of the text, so a timed one missing a
+// segment's words would lose them; the caller then stores the whole text untimed.
+// Speakers become the meeting path's ids, in order of first appearance, so the
+// UI translates their labels, renaming and mobile key on them, and a recording
+// made into the note later numbers its own speakers after them.
+test("provider speaker labels become meeting speaker ids", async () => {
+ const { buildUploadTranscript } = await load();
+
+ const stored = JSON.parse(
+ buildUploadTranscript(
+ [
+ { text: "One.", start: 0, speaker: "B" },
+ { text: "Two.", start: 1, speaker: "A" },
+ { text: "Three.", start: 2, speaker: "B" },
+ { text: "Four.", start: 3 },
+ ],
+ 1_700_000_000_000
+ )
+ );
+ assert.deepEqual(
+ stored.map((segment) => segment.speaker),
+ ["speaker_0", "speaker_1", "speaker_0", undefined]
+ );
+ assert.ok(stored.every((segment) => !("speakerName" in segment)));
+});
+
+test("a spoken segment without a start gives no timed transcript", async () => {
+ const { buildUploadTranscript } = await load();
+
+ assert.equal(
+ buildUploadTranscript([
+ { text: "First sentence.", start: 0 },
+ { text: "Second sentence.", start: NaN },
]),
- });
+ undefined
+ );
+ assert.equal(
+ JSON.parse(
+ buildUploadTranscript(
+ [
+ { text: " ", start: NaN },
+ { text: "Kept.", start: 2 },
+ ],
+ 1_700_000_000_000
+ )
+ ).length,
+ 1,
+ "a blank segment carries no words, so its missing start loses nothing"
+ );
});
test("upload titles fall back to the transcript, then the file name", async () => {
diff --git a/test/services/uploadTranscriptPersistence.test.js b/test/services/uploadTranscriptPersistence.test.js
new file mode 100644
index 000000000..496611ef7
--- /dev/null
+++ b/test/services/uploadTranscriptPersistence.test.js
@@ -0,0 +1,330 @@
+const test = require("node:test");
+const assert = require("node:assert/strict");
+const fs = require("node:fs");
+const path = require("node:path");
+const { createDb } = require("../helpers/harness/db");
+const { createRendererServer, installBrowserGlobals } = require("../lib/rendererTestHarness");
+
+for (const source of ["file", "url"]) {
+ for (const timed of [false, true]) {
+ test(`${source} upload stores ${timed ? "timed" : "text-only"} transcription separately at creation`, async (t) => {
+ const db = createDb(t);
+ assert.ok(db);
+ const handwritten = db.saveNote("My notes", "Keep my handwritten notes").note;
+ const meeting = db.saveNote("Existing meeting", "Meeting notes", "meeting").note;
+ db.updateNote(meeting.id, { transcript: "Existing transcript", enhanced_content: "Summary" });
+ const originals = [db.getNote(handwritten.id), db.getNote(meeting.id)];
+ const text = "Public synthetic transcript.\nSecond paragraph.";
+ const segments = timed ? [{ text, start: 2, end: 5, speaker: "Speaker 1" }] : undefined;
+ const created = [];
+ const { window } = installBrowserGlobals(t);
+ Object.assign(window.electronAPI, {
+ transcribeAudioFile: async () => ({ success: true, text, segments, durationSeconds: 12 }),
+ downloadUrlAudio: async () => ({
+ success: true,
+ tempPath: "/synthetic/file.wav",
+ title: "Public title",
+ sizeBytes: 100,
+ }),
+ onUrlDownloadProgress: () => () => {},
+ saveNote: async (...args) => {
+ const result = db.saveNote(...args);
+ created.push(result.note);
+ return result;
+ },
+ // Metadata failure must never lose the already-created transcript.
+ updateNote: async () => {
+ throw new Error("metadata unavailable");
+ },
+ deleteTempFile: () => {},
+ });
+ const vite = await createRendererServer(t, {
+ mockModules: {
+ "/lib/auth": "export const withSessionRefresh = (fn) => fn();",
+ "./settingsStore": "export const getSettings = () => ({});",
+ "./policyStore": "export const usePolicyStore = { getState: () => ({}) };",
+ "./policyRules": "export const isTranscriptionContextAllowed = () => true;",
+ },
+ });
+ const queue = await vite.ssrLoadModule("/stores/batchQueueStore.ts");
+ if (source === "file")
+ queue.addFiles([{ name: "fixture.wav", path: "/synthetic/file.wav", sizeBytes: 100 }]);
+ else queue.addUrls(["https://www.youtube.com/watch?v=public123"]);
+ queue.processBatchQueue(
+ {
+ transcription: {
+ useLocalWhisper: true,
+ localTranscriptionProvider: "whisper",
+ whisperModel: "base",
+ parakeetModel: "unused",
+ isOpenWhisprCloud: false,
+ getApiKey: () => "",
+ cloudTranscriptionProvider: "openai",
+ cloudTranscriptionBaseUrl: "",
+ cloudTranscriptionModel: "whisper-1",
+ language: "en",
+ },
+ folderId: null,
+ generateTitle: async () => "Generated title",
+ },
+ { enabled: timed, localModelsReady: false, numSpeakers: null }
+ );
+ const deadline = Date.now() + 5000;
+ while (queue.useBatchQueueStore.getState().isProcessing && Date.now() < deadline) {
+ await new Promise((resolve) => setTimeout(resolve, 10));
+ }
+ const item = queue.useBatchQueueStore.getState().queue[0];
+ assert.equal(item.status, "done");
+ const note = db.getNote(item.noteId);
+ assert.equal(note.content, "");
+ assert.equal(
+ created[0].transcript,
+ note.transcript,
+ "initial note-added/mirror snapshot includes transcript"
+ );
+ assert.deepEqual(
+ JSON.parse(note.transcript).map((segment) => segment.text),
+ [text]
+ );
+ assert.equal(JSON.parse(note.transcript)[0].speaker, timed ? "speaker_0" : undefined);
+ assert.equal(typeof JSON.parse(note.transcript)[0].timestamp, timed ? "number" : "undefined");
+ assert.equal(note.note_type, "upload");
+ assert.equal(note.title, source === "file" ? "Generated title" : "Public title");
+ assert.equal(note.source_file, source === "file" ? "fixture.wav" : "Public title");
+ assert.deepEqual([db.getNote(handwritten.id), db.getNote(meeting.id)], originals);
+ const filename = db.db.name;
+ db.db.close();
+ db.db = new (require("better-sqlite3"))(filename);
+ assert.equal(db.getNote(note.id).transcript, note.transcript);
+ const mirror = require("../../src/helpers/markdownMirror");
+ const mirrorPath = path.join(path.dirname(filename), "mirror");
+ mirror.init(mirrorPath);
+ mirror.writeNote(note, "Personal");
+ mirror.writeTranscript(note, "Personal", {});
+ const transcriptPath = fs
+ .readdirSync(path.join(mirrorPath, "Personal"))
+ .find((name) => name.endsWith("-transcript.md"));
+ assert.ok(transcriptPath);
+ assert.ok(
+ fs.readFileSync(path.join(mirrorPath, "Personal", transcriptPath), "utf8").includes(text)
+ );
+ });
+ }
+}
+
+test("upload transcript crosses the actual preload and save IPC handler before broadcasting", async (t) => {
+ const db = createDb(t);
+ assert.ok(db);
+ const vm = require("node:vm");
+ const ts = require("typescript");
+ const source = ts.createSourceFile(
+ "ipcHandlers.js",
+ fs.readFileSync("src/helpers/ipcHandlers.js", "utf8"),
+ ts.ScriptTarget.Latest,
+ true
+ );
+ let callback;
+ function visit(node) {
+ if (ts.isCallExpression(node) && node.arguments[0]?.text === "db-save-note") {
+ callback = node.arguments[1].getText(source);
+ }
+ ts.forEachChild(node, visit);
+ }
+ visit(source);
+ assert.ok(callback);
+ const broadcasts = [];
+ const mirrored = [];
+ const owner = {
+ databaseManager: db,
+ notifyVectorChanges() {},
+ _asyncMirrorWrite: (note) => mirrored.push(note),
+ };
+ const handler = vm.runInNewContext(`(function () { return (${callback}); }).call(owner)`, {
+ owner,
+ setImmediate: (fn) => fn(),
+ broadcastToWindows: (...args) => broadcasts.push(args),
+ });
+ const { window } = installBrowserGlobals(t);
+ vm.runInNewContext(fs.readFileSync("preload.js", "utf8"), {
+ process,
+ require: (name) => {
+ assert.equal(name, "electron");
+ return {
+ contextBridge: {
+ exposeInMainWorld: (_name, api) => {
+ window.electronAPI = api;
+ },
+ },
+ webUtils: {},
+ ipcRenderer: {
+ invoke: async (channel, ...args) => {
+ assert.equal(channel, "db-save-note");
+ return handler.call(owner, {}, ...args);
+ },
+ on() {},
+ removeListener() {},
+ send() {},
+ sendSync() {},
+ },
+ };
+ },
+ });
+ const { saveUploadNote } = await import("../../src/services/uploadNotes.ts");
+ const saved = await saveUploadNote({
+ title: "Atomic upload",
+ text: "Public fixture words",
+ sourceName: "fixture.wav",
+ folderId: null,
+ diarization: { enabled: false, localModelsReady: false, numSpeakers: null },
+ });
+ const stored = db.getNote(saved.note.id);
+ assert.equal(stored.content, "");
+ assert.deepEqual(JSON.parse(stored.transcript), [{ text: "Public fixture words" }]);
+ assert.equal(broadcasts[0][0], "note-added");
+ assert.equal(broadcasts[0][1].transcript, stored.transcript);
+ assert.equal(mirrored[0].transcript, stored.transcript);
+});
+
+test("a segment without a start keeps the complete uploaded text without invented times", async (t) => {
+ const db = createDb(t);
+ assert.ok(db);
+ const { window } = installBrowserGlobals(t);
+ window.electronAPI.saveNote = async (...args) => db.saveNote(...args);
+ const { saveUploadNote } = await import("../../src/services/uploadNotes.ts");
+ const text = "First sentence. Second sentence.";
+ const saved = await saveUploadNote({
+ title: "Full result",
+ text,
+ sourceName: "fixture.wav",
+ folderId: null,
+ segments: [
+ { text: "First sentence.", start: 0 },
+ { text: "Second sentence.", start: NaN },
+ ],
+ diarization: { enabled: false, localModelsReady: false, numSpeakers: null },
+ });
+ assert.equal(saved.note.content, "");
+ assert.deepEqual(JSON.parse(db.getNote(saved.note.id).transcript), [{ text }]);
+});
+
+// The provider's text is formatted for reading (speaker headers) or joined
+// without spaces (CJK), so it never equals the segments' text; the segments
+// still hold the words, and their timing and speakers must survive.
+for (const { name, text, segments, speakers } of [
+ {
+ name: "a provider-diarized upload keeps its speakers and times",
+ text: "[A] 00:00 - 00:02\nhello there\n\n[B] 00:02 - 00:04\nhi back",
+ segments: [
+ { speaker: "A", text: "hello there", start: 0, end: 2 },
+ { speaker: "B", text: "hi back", start: 2, end: 4 },
+ ],
+ speakers: ["speaker_0", "speaker_1"],
+ },
+ {
+ name: "a CJK upload keeps its segment times",
+ text: "你好。今天开会。",
+ segments: [
+ { text: "你好。", start: 0, end: 1 },
+ { text: "今天开会。", start: 1, end: 2 },
+ ],
+ speakers: [undefined, undefined],
+ },
+]) {
+ test(name, async (t) => {
+ const db = createDb(t);
+ assert.ok(db);
+ const { window } = installBrowserGlobals(t);
+ window.electronAPI.saveNote = async (...args) => db.saveNote(...args);
+ const { saveUploadNote } = await import("../../src/services/uploadNotes.ts");
+ const saved = await saveUploadNote({
+ title: "Timed",
+ text,
+ sourceName: "fixture.wav",
+ folderId: null,
+ segments,
+ diarization: { enabled: false, localModelsReady: false, numSpeakers: null },
+ });
+ const stored = JSON.parse(db.getNote(saved.note.id).transcript);
+ assert.deepEqual(
+ stored.map((segment) => segment.text),
+ segments.map((segment) => segment.text)
+ );
+ assert.deepEqual(
+ stored.map((segment) => segment.speaker),
+ speakers
+ );
+ assert.equal(
+ stored[1].timestamp - stored[0].timestamp,
+ (segments[1].start - segments[0].start) * 1000
+ );
+ });
+}
+
+test("upload creation keeps its selected workspace and sends transcript in the real cloud payload", async (t) => {
+ const db = createDb(t);
+ assert.ok(db);
+ db.setActiveAccountId("synthetic-account");
+ const space = db.db
+ .prepare(
+ "INSERT INTO spaces (client_space_id, kind, name, sort_order) VALUES (?, 'team', ?, 1)"
+ )
+ .run("synthetic-space", "Target workspace");
+ const spaceId = Number(space.lastInsertRowid);
+ db.db
+ .prepare("INSERT INTO space_accounts (space_id, account_id) VALUES (?, ?)")
+ .run(spaceId, "synthetic-account");
+ const folder = db.createFolder("Target folder", spaceId).folder;
+ assert.ok(folder);
+ const calls = [];
+ const { window } = installBrowserGlobals(t);
+ Object.assign(window.electronAPI, {
+ saveNote: async (...args) => db.saveNote(...args),
+ cloudApiRequest: async (request) => {
+ assert.equal(request.path, "/api/notes/create");
+ calls.push(JSON.parse(JSON.stringify(request)));
+ return {
+ success: true,
+ data: {
+ ...request.body,
+ id: "synthetic-cloud-note",
+ updated_at: "2026-10-22T00:00:00Z",
+ user_id: "synthetic-account",
+ row_created: true,
+ write_applied: true,
+ },
+ };
+ },
+ acknowledgeNoteCreate: async (...args) => db.acknowledgeNoteCreate(...args),
+ getNote: async (id) => db.getNote(id),
+ });
+ const vite = await createRendererServer(t);
+ const { saveUploadNote } = await vite.ssrLoadModule("/services/uploadNotes.ts");
+ const text = "Full uploaded words for cloud backup.";
+ const saved = await saveUploadNote({
+ title: "Scoped upload",
+ text,
+ sourceName: "fixture.wav",
+ folderId: folder.id,
+ diarization: { enabled: false, localModelsReady: false, numSpeakers: null },
+ });
+ const note = db.getNote(saved.note.id);
+ assert.equal(note.folder_id, folder.id);
+ assert.equal(note.space_id, spaceId);
+ const { SyncService } = await vite.ssrLoadModule("/services/SyncService.ts");
+ const service = new SyncService();
+ await service.createCloudNote(note, "synthetic-cloud-folder", {
+ space_id: "synthetic-cloud-space",
+ });
+ assert.equal(calls.length, 1);
+ const { body } = calls[0];
+ assert.equal(body.content, "");
+ assert.equal(body.transcript, note.transcript);
+ assert.deepEqual(JSON.parse(body.transcript), [{ text }]);
+ assert.equal(body.folder_id, "synthetic-cloud-folder");
+ assert.equal(body.space_id, "synthetic-cloud-space");
+ const { buildNoteUpdatePayload } = await vite.ssrLoadModule("/helpers/cloudSyncGuards.js");
+ const update = JSON.parse(
+ JSON.stringify(buildNoteUpdatePayload({ ...note, cloud_revision: 3 }, "synthetic-cloud-folder"))
+ );
+ assert.equal(update.transcript, note.transcript);
+});