Skip to content

Voice-to-text dictation app with local (Nvidia Parakeet/Whisper) and cloud models (BYOK). Privacy-first and available cross-platform.

chore(agent): drop noteChat and Mem0 leftovers now that the API has no memory (#2630)

Joshuac0d0490fe5fb
diff --git a/CLAUDE.md b/CLAUDE.md
index 299a690ec..fc77c5245 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -731,7 +731,7 @@ A dedicated global hotkey that starts a dictation whose transcript is sent strai
4. Standalone commands (no text selected) run through the chat pipeline (`src/components/dictation/AssistantPanel.tsx`): chat tools (notes search/create/update, calendar, web search, clipboard, `get_snippet` — triggers listed in the tool description, body fetched on demand — and `update_dictionary` / `update_snippets`, which write through the settings store so the change syncs like a UI edit), RAG memory, and the custom dictionary plus snippet triggers injected into the system prompt. Conversations persist in the `agent_conversations` table and are browsable from the ControlPanel chat. The system prompt (`getAgentSystemPrompt`, appended after the user-editable base prompt, so it reaches custom prompts too) lists the offered tools grouped by capability, forbids claiming an ability a listed tool covers (web search is for public information the context doesn't already answer, never people the user knows), and names what is unavailable and how to enable it — for an unavailable connector the model still writes the email or message in its reply and ends with one line on how to have it sent (`resolveUnavailableCapabilities` in `src/config/agentCapabilities.ts`: signed out, org policy, paid plan, a Slack/Linear/GitHub connector not connected or needing a reconnect, no calendar, or a local model too small for tools; connectors are named only on surfaces that offer them, and the onboarding demo names nothing, `nameUnavailableCapabilities: false`). History (`toHistoryMessages` in `src/components/chat/historyMessages.ts`) prefixes earlier assistant turns with `[Tools used: …]`, showing only allowlisted search queries, contact names and note titles — never drafted content or results. A connector action adds its outcome from the saved result (`sent`, `draft opened`, `may have been sent`, `cancelled by the user`, `not sent`, `failed`, `unavailable`, `needed details`), and a sent one its reference and an `https` link (`github_create_issue (sent: o/r#12 https://…)`), so a follow-up turn ("it's #engineering", "retry") never redoes what already went out; destinations, card edits and error text stay out. A call cut off before its result reads "(outcome not recorded)", since a send can still commit after Esc. A reply that opens with such a note itself is shown, saved and delivered without it (`withoutEchoedToolTrace`)
5. Response delivery: a capture with `status: "editable"` (a focused writable non-terminal field with no selection) plus auto-paste banks a `deliverySessionId`; the completed answer is pasted via `paste-at-captured-target`, which revalidates the target and fails closed to the panel + clipboard on any change (`assistantResponseDelivery.ts`, `pasteAtCapturedTarget` in `selectionManager.js`). A follow-up spoken while the panel is already open stays panel-first. Cancelled or empty responses never paste and never touch the clipboard
6. Selection edits: highlighted text goes through the `dictationAgent` scope and is safely replaced in place. When the editor judges the command a question about the selection ("explain this") rather than a change, it answers `SELECTION_QUESTION_MARKER` (`selectionEditing.js`, both the cloud and local prompts) and the command goes to the panel with the selection quoted, leaving the selection untouched
-7. Connector tools (`email_draft`, `find_contact`, `slack_send_message`, the Linear tools, the GitHub tools; paid, policy-gated) run through `src/helpers/connectors`. Typed chat (`ChatView`), this panel and every note's chat (`useEmbeddedChat`) opt in (`allowConnectors`); container chat and the onboarding demo don't get them. Note chat also gets a "Meeting attendees" block: the note chat sends its meeting as one request (`NoteAttendeesRequest`: note id, parsed participants, calendar event id, the user's OpenWhispr address), and main adds the speakers identified in the note (speaker profiles with an address) and the linked calendar event's organizer (Outlook and Apple leave them out of the attendees), keeps only valid addresses, strips control and invisible format characters, angle brackets and their look-alikes, and address-like words from names, and filters with `find_contact`'s exclusions plus the user's OpenWhispr address and the Gmail login's address (`connector-note-attendees`, `createNoteAttendeesLookup`), organizer included. The block is fenced in `<meeting_attendees>` tags and marked as data, and is added only when connector tools are registered; the note's title and text and the `search_notes` and `get_note` results lose the tag name (`withoutAttendeesFence`), so an invite's title can't fake a second list. A participant flagged `self` also excludes that address when it comes back as an identified speaker or the organizer. A participant's `self` flag marks whoever recorded the meeting, so the renderer honours it only on the user's own note (`ownsNote`, `attendeesForUser`; unknown ownership counts as someone else's). The note chat passes its note as `openNote`, which `getAgentSystemPrompt` puts under its own rule: for questions about what was said, decided or written, answer from this note, and when it doesn't cover the question, say so and offer to search the user's other notes; `search_notes` runs only when the user asks to look beyond the note, and the answer names the note used. Unlike the other chats it runs no up-front library search (RAG): other notes reach it only through `search_notes` and `get_note`, so a model too small for tools answers from the open note alone. The open note carries its transcript whole, as `Speaker: text` lines labelled exactly as `get_note` labels them (`noteChatTranscript`: saved speaker names, `Note taker` for whoever recorded it, numbered in-person speakers; after a line saying whether the note taker is the user, named on their own note), unless the model has `get_note` and the transcript is longer than it takes (`FULL_TRANSCRIPT_MAX_TOKENS` in `transcriptEvidence.ts`, counted with `estimateNoteTokens` since CJK text is about a token per character: 64,000 hosted, 4,000 on a local or LAN model); then it carries the first 500 characters (`transcript_preview`) and a rule to read further with `get_note` before saying the note doesn't cover something. Whole-note actions (`requestText`) always get the whole transcript. `get_note` returns the start of the transcript beside the editable fields; `transcript_query` (literal and case-insensitive, speaker names included; the person taking the notes is `Note taker`) or a `transcript_offset` past 0 returns one 500-character passage with the times it spans, paging metadata (offset 0 is an ordinary read, since models fill in defaults) and a revision naming the text read so far, so a recording that keeps appending can still be paged. Both tool loops keep the two newest passages, and every passage of the latest step even when it read more in parallel (`compactTranscriptHistory`); a query that matched nothing is never compacted. Its OpenWhispr Cloud requests carry `noteChat: true`, so the API puts their mem0 memories under a heading that marks them as past conversations, for who the user is rather than facts about the note (#2551). Like `ChatView`, New chat and switching conversations cancel the running turn, so a pending card can't hold the send lock or save into another conversation. An approval card, any `email_draft` (its compose window takes focus), any `find_contact` (a lookup is almost always followed by a question for the user), and any `slack_send_message`, `linear_search_issues`, `linear_create_issue`, `linear_comment` or GitHub tool (holds on every outcome — a card, a question back, results or a receipt) hold the turn in the panel (`onHoldDelivery`) instead of pasting at the caret. A held answer is still auto-copied, except after an `email_draft` that reserved the clipboard for overflow text (`onHoldDelivery({ preserveClipboard: true })`, sticky for the turn) and on a caret turn whose delivery restores the clipboard (Keep Transcription in Clipboard off)
+7. Connector tools (`email_draft`, `find_contact`, `slack_send_message`, the Linear tools, the GitHub tools; paid, policy-gated) run through `src/helpers/connectors`. Typed chat (`ChatView`), this panel and every note's chat (`useEmbeddedChat`) opt in (`allowConnectors`); container chat and the onboarding demo don't get them. Note chat also gets a "Meeting attendees" block: the note chat sends its meeting as one request (`NoteAttendeesRequest`: note id, parsed participants, calendar event id, the user's OpenWhispr address), and main adds the speakers identified in the note (speaker profiles with an address) and the linked calendar event's organizer (Outlook and Apple leave them out of the attendees), keeps only valid addresses, strips control and invisible format characters, angle brackets and their look-alikes, and address-like words from names, and filters with `find_contact`'s exclusions plus the user's OpenWhispr address and the Gmail login's address (`connector-note-attendees`, `createNoteAttendeesLookup`), organizer included. The block is fenced in `<meeting_attendees>` tags and marked as data, and is added only when connector tools are registered; the note's title and text and the `search_notes` and `get_note` results lose the tag name (`withoutAttendeesFence`), so an invite's title can't fake a second list. A participant flagged `self` also excludes that address when it comes back as an identified speaker or the organizer. A participant's `self` flag marks whoever recorded the meeting, so the renderer honours it only on the user's own note (`ownsNote`, `attendeesForUser`; unknown ownership counts as someone else's). The note chat passes its note as `openNote`, which `getAgentSystemPrompt` puts under its own rule: for questions about what was said, decided or written, answer from this note, and when it doesn't cover the question, say so and offer to search the user's other notes; `search_notes` runs only when the user asks to look beyond the note, and the answer names the note used. Unlike the other chats it runs no up-front library search (RAG): other notes reach it only through `search_notes` and `get_note`, so a model too small for tools answers from the open note alone. The open note carries its transcript whole, as `Speaker: text` lines labelled exactly as `get_note` labels them (`noteChatTranscript`: saved speaker names, `Note taker` for whoever recorded it, numbered in-person speakers; after a line saying whether the note taker is the user, named on their own note), unless the model has `get_note` and the transcript is longer than it takes (`FULL_TRANSCRIPT_MAX_TOKENS` in `transcriptEvidence.ts`, counted with `estimateNoteTokens` since CJK text is about a token per character: 64,000 hosted, 4,000 on a local or LAN model); then it carries the first 500 characters (`transcript_preview`) and a rule to read further with `get_note` before saying the note doesn't cover something. Whole-note actions (`requestText`) always get the whole transcript. `get_note` returns the start of the transcript beside the editable fields; `transcript_query` (literal and case-insensitive, speaker names included; the person taking the notes is `Note taker`) or a `transcript_offset` past 0 returns one 500-character passage with the times it spans, paging metadata (offset 0 is an ordinary read, since models fill in defaults) and a revision naming the text read so far, so a recording that keeps appending can still be paged. Both tool loops keep the two newest passages, and every passage of the latest step even when it read more in parallel (`compactTranscriptHistory`); a query that matched nothing is never compacted. Its OpenWhispr Cloud requests are ordinary `/api/agent/stream` requests: the API keeps no assistant memory and adds nothing to the prompt, so the client's own system prompt is what the model sees (the `noteChat` flag went once openwhispr-api#243 removed memory). Like `ChatView`, New chat and switching conversations cancel the running turn, so a pending card can't hold the send lock or save into another conversation. An approval card, any `email_draft` (its compose window takes focus), any `find_contact` (a lookup is almost always followed by a question for the user), and any `slack_send_message`, `linear_search_issues`, `linear_create_issue`, `linear_comment` or GitHub tool (holds on every outcome — a card, a question back, results or a receipt) hold the turn in the panel (`onHoldDelivery`) instead of pasting at the caret. A held answer is still auto-copied, except after an `email_draft` that reserved the clipboard for overflow text (`onHoldDelivery({ preserveClipboard: true })`, sticky for the turn) and on a caret turn whose delivery restores the clipboard (Keep Transcription in Clipboard off)
**Storage & IPC**:
diff --git a/src/components/chat/useChatStreaming.ts b/src/components/chat/useChatStreaming.ts
index a3e3d6989..006c37161 100644
--- a/src/components/chat/useChatStreaming.ts
+++ b/src/components/chat/useChatStreaming.ts
@@ -663,8 +663,6 @@ export function useChatStreaming({
})),
executeToolCall,
...(cloudScreenContext ? { screenContext: cloudScreenContext } : {}),
- // So the server marks its memories as past conversations, not facts about the note.
- ...(openNoteRef.current ? { noteChat: true } : {}),
});
} else {
const aiTools = registry?.toAISDKFormat(
diff --git a/src/helpers/ipcHandlers.js b/src/helpers/ipcHandlers.js
index 8ae5a26a4..a564b97f5 100644
--- a/src/helpers/ipcHandlers.js
+++ b/src/helpers/ipcHandlers.js
@@ -10160,7 +10160,6 @@ class IPCHandlers {
systemPrompt: opts.systemPrompt,
tools: opts.tools,
...(opts.screenContext ? { screenContext: opts.screenContext } : {}),
- ...(opts.noteChat ? { noteChat: true } : {}),
sessionId: this.sessionId,
clientType: "desktop",
appVersion: app.getVersion(),
diff --git a/src/services/ReasoningService.ts b/src/services/ReasoningService.ts
index 0b6caf7be..baa44c356 100644
--- a/src/services/ReasoningService.ts
+++ b/src/services/ReasoningService.ts
@@ -1123,7 +1123,6 @@ class ReasoningService extends BaseReasoningService {
tools?: Array<{ name: string; description: string; parameters: Record<string, unknown> }>;
// Press-time screenshot; the server routes to its vision chain when present.
screenContext?: { data: string; mediaType: string };
- noteChat?: boolean;
}
): {
stream: AsyncGenerator<
@@ -1230,7 +1229,6 @@ class ReasoningService extends BaseReasoningService {
toolCallId: string
) => Promise<ToolExecutionResult>;
screenContext?: { data: string; mediaType: string };
- noteChat?: boolean;
}
): AsyncGenerator<AgentStreamChunk, void, unknown> {
// Capture synchronously so a cancel before the first next() is observed.
@@ -1249,7 +1247,6 @@ class ReasoningService extends BaseReasoningService {
toolCallId: string
) => Promise<ToolExecutionResult>;
screenContext?: { data: string; mediaType: string };
- noteChat?: boolean;
},
operationGeneration: number
): AsyncGenerator<AgentStreamChunk, void, unknown> {
@@ -1275,7 +1272,6 @@ class ReasoningService extends BaseReasoningService {
systemPrompt: config.systemPrompt,
tools: config.tools,
screenContext: config.screenContext,
- noteChat: config.noteChat,
});
const pendingToolCalls: Array<{ id: string; name: string; arguments: string }> = [];
diff --git a/src/types/electron.ts b/src/types/electron.ts
index 53546c23b..fd9b2b096 100644
--- a/src/types/electron.ts
+++ b/src/types/electron.ts
@@ -3117,7 +3117,6 @@ declare global {
systemPrompt?: string;
tools?: Array<{ name: string; description: string; parameters: Record<string, unknown> }>;
screenContext?: { data: string; mediaType: string };
- noteChat?: boolean;
}
) => void;
cancelAgentStream?: (requestId: string) => void;
diff --git a/src/types/policy.ts b/src/types/policy.ts
index e48f70e68..18271abfa 100644
--- a/src/types/policy.ts
+++ b/src/types/policy.ts
@@ -42,8 +42,9 @@ export interface OrgPolicy {
*/
connectorsEnabled?: boolean;
/**
- * Server-only Mem0 agent-memory gate, enforced by the API on
- * `/api/agent/stream`; the app never reads it. Absent on older servers.
+ * Retired agent-memory gate: the API stopped using it when it removed
+ * assistant memory (openwhispr-api#243), but older API versions may still
+ * send it. The app ignores it.
*/
memoryEnabled?: boolean;
};
diff --git a/test/helpers/policyValidation.test.js b/test/helpers/policyValidation.test.js
index b8d3122b1..e9cbc58b5 100644
--- a/test/helpers/policyValidation.test.js
+++ b/test/helpers/policyValidation.test.js
@@ -92,9 +92,9 @@ test("rejects a malformed requiredLocalModels", () => {
}
});
-test("tolerates the server-only memoryEnabled flag", () => {
- // The API gates Mem0 agent memory itself and the app never reads the field,
- // so its presence must not invalidate an otherwise valid policy.
+test("tolerates the retired memoryEnabled flag", () => {
+ // Older API versions still send it after memory was removed
+ // (openwhispr-api#243); the app ignores it, so it must not invalidate a policy.
for (const value of [true, false]) {
const policy = validPolicy();
policy.features.memoryEnabled = value;
diff --git a/test/helpers/useChatStreamingTools.test.js b/test/helpers/useChatStreamingTools.test.js
index dcf0a5906..b62c7f7cf 100644
--- a/test/helpers/useChatStreamingTools.test.js
+++ b/test/helpers/useChatStreamingTools.test.js
@@ -723,8 +723,8 @@ test("a note's chat answers from its note and adds no other notes up front", asy
await captured.sendToAI("What price are we testing?", []);
assert.deepEqual(searches, []);
- // So the server marks its memories as past conversations.
- assert.equal(configs[0].noteChat, true);
+ // The API no longer keeps memories, so a note chat's request is an ordinary one.
+ assert.equal("noteChat" in configs[0], false);
assert.match(prompts[0], /inside the note below\.[^\n]*\n\nNote ID: 7\nTitle: Kickoff/);
assert.doesNotMatch(prompts[0], /notes from the user's library/);
});
@@ -739,7 +739,7 @@ test("a chat that isn't about a note still lists the library notes a question fi
await captured.sendToAI("What price are we testing?", []);
assert.deepEqual(searches, ["What price are we testing?"]);
- assert.equal(configs[0].noteChat, undefined);
+ assert.equal("noteChat" in configs[0], false);
assert.match(
prompts[0],
/notes from the user's library[^]*<note id="3" title="Pricing sync Sep 29">\nTesting \$15 against \$12/
diff --git a/test/hooks/noteEditSurfaces.test.js b/test/hooks/noteEditSurfaces.test.js
index aa7338e00..46be909e0 100644
--- a/test/hooks/noteEditSurfaces.test.js
+++ b/test/hooks/noteEditSurfaces.test.js
@@ -244,7 +244,7 @@ for (const surface of ["note chat", "assistant panel"]) {
assert.match(JSON.stringify(request.messages), /email_draft \(draft opened\)/);
if (surface === "note chat") {
assert.match(request.config.systemPrompt, /Note fields \(JSON\)/);
- if (provider === "cloud") assert.equal(request.config.noteChat, true);
+ assert.equal("noteChat" in request.config, false);
} else if (provider === "byok")
assert.equal(request.config.inferenceScope, "dictationAgent");
});
diff --git a/test/services/reasoningServiceStreamingThink.test.js b/test/services/reasoningServiceStreamingThink.test.js
index 9e4931949..dfae2423d 100644
--- a/test/services/reasoningServiceStreamingThink.test.js
+++ b/test/services/reasoningServiceStreamingThink.test.js
@@ -656,7 +656,6 @@ test("cloud action requests supply source fidelity once through every tool step"
systemPrompt,
tools: [{ name: "get_note", description: "Read a note", parameters: {} }],
executeToolCall: async () => ({ data: "Not yet approved.", displayText: "Read note" }),
- noteChat: true,
})
);
await waitForMicrotasks();
@@ -678,7 +677,6 @@ test("cloud action requests supply source fidelity once through every tool step"
const effective = [{ role: "system", content: config.systemPrompt }, ...sent];
assert.equal(JSON.stringify(effective).split(SOURCE_FIDELITY_RULE).length - 1, 1);
assert.equal(config.systemPrompt, systemPrompt);
- assert.equal(config.noteChat, true);
assert.equal(sent[0].content, action);
}
assert.deepEqual(
@@ -756,19 +754,26 @@ test("cloud agent streaming correlates events to the initiating request", async
assert.deepEqual(bridge.cleanupCounts, { chunk: 1, error: 1, end: 1 });
});
-test("a note chat's cloud stream tells main it is a note chat", async (t) => {
+test("a cloud stream sends main only the options the API request carries", async (t) => {
const bridge = createAgentStreamBridge();
- const { reasoningService } = await loadReasoningService(t, "openwhispr-cloud-agent-note-chat-", {
+ const { reasoningService } = await loadReasoningService(t, "openwhispr-cloud-agent-options-", {
window: { electronAPI: bridge.electronAPI },
});
const stream = reasoningService.processTextStreamingCloud([{ role: "user", content: "hello" }], {
systemPrompt: "Answer the user.",
+ tools: [{ name: "get_note", description: "Read a note", parameters: {} }],
+ screenContext: { data: "aGVsbG8=", mediaType: "image/jpeg" },
+ // Retired with assistant memory (openwhispr-api#243); must not reach main.
noteChat: true,
});
const pending = stream.next();
await waitForMicrotasks();
- assert.equal(bridge.startCalls[0][2].noteChat, true);
+ assert.deepEqual(Object.keys(bridge.startCalls[0][2]).sort(), [
+ "screenContext",
+ "systemPrompt",
+ "tools",
+ ]);
reasoningService.cancelActiveStream();
await pending;
});