Skip to content

Voice-to-text dictation app with local (Nvidia Parakeet/Whisper) and cloud models (BYOK). Privacy-first and available cross-platform.

feat: added in agent capabilities

Gabriel Stein49b98c4da525
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c2f078fe6..161ec67f2 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
+- **Agent Naming System**: Personalize your AI assistant with a custom name for more natural interactions
+ - Name your agent during onboarding (step 6 of 8)
+ - Address your agent directly: "Hey [AgentName], make this more professional"
+ - Update agent name anytime through settings
+ - Smart AI processing distinguishes between commands and regular dictation
+ - Clean output automatically removes agent name references
- **Draggable Interface**: Click and drag the dictation panel to any position on screen
- **Dynamic Hotkey Display**: Tooltip shows your actual hotkey setting instead of generic text
- **Flexible Hotkey System**: Fixed hardcoded hotkey limitation - now fully respects user settings
@@ -24,6 +30,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **User Experience**: Eliminated confusion between drag and click actions
### Technical Details
+- **Agent Naming Implementation**:
+ - Added centralized agent name utility (`src/utils/agentName.ts`)
+ - Enhanced onboarding flow with agent naming step
+ - Updated ReasoningService with context-aware AI processing
+ - Added agent name settings section with comprehensive UI
+ - Implemented smart prompt generation for agent-addressed vs regular text
- Added IPC handlers for dynamic hotkey updates (`update-hotkey`)
- Implemented window-level dragging using screen cursor tracking
- Added real-time hotkey loading from localStorage in main dictation component
diff --git a/README.md b/README.md
index 3628591e8..a5ba96f83 100644
--- a/README.md
+++ b/README.md
@@ -6,6 +6,7 @@ An open source desktop dictation application that converts speech to text using
- 🎀 **Global Hotkey**: Customizable hotkey to start/stop dictation from anywhere (default: backtick `)
- πŸ€– **Dual AI Processing**: Choose between local Whisper models (private) or OpenAI API (fast)
+- 🎯 **Agent Naming**: Personalize your AI assistant with a custom name for natural interactions
- πŸ”’ **Privacy-First**: Local processing keeps your voice data completely private
- 🎨 **Modern UI**: Built with React 19, TypeScript, and Tailwind CSS v4
- πŸš€ **Fast**: Optimized with Vite and modern tooling
@@ -59,7 +60,12 @@ An open source desktop dictation application that converts speech to text using
- **Microphone Access**: Required for voice recording
- **Accessibility Permissions**: Required for automatic text pasting (macOS)
-3. **Configure Global Hotkey**: Default is backtick (`) but can be customized
+3. **Name Your Agent**: Give your AI assistant a personal name (e.g., "Assistant", "Jarvis", "Alex")
+ - Makes interactions feel more natural and conversational
+ - Helps distinguish between giving commands and regular dictation
+ - Can be changed anytime in settings
+
+4. **Configure Global Hotkey**: Default is backtick (`) but can be customized
## Usage
@@ -77,6 +83,22 @@ An open source desktop dictation application that converts speech to text using
- **Models**: Download and manage local Whisper models
- **Settings**: Configure API keys, customize hotkeys, and manage permissions
+### Agent Naming
+Once you've named your agent during setup, you can interact with it in two ways:
+
+**🎯 Agent Commands** (for AI assistance):
+- "Hey [AgentName], make this more professional"
+- "Hey [AgentName], format this as a list"
+- "Hey [AgentName], write a thank you email"
+- "Hey [AgentName], convert this to bullet points"
+
+**πŸ“ Regular Dictation** (for normal text):
+- "This is just normal text I want transcribed"
+- "Meeting notes: John mentioned the quarterly report"
+- "Dear Sarah, thank you for your help"
+
+The AI automatically detects when you're giving it commands versus dictating regular text, and removes agent name references from the final output.
+
### Processing Options
- **Local Processing**:
- Install Whisper automatically through the Control Panel
@@ -106,6 +128,8 @@ open-wispr/
β”‚ β”œβ”€β”€ vite.config.js # Vite configuration
β”‚ β”œβ”€β”€ components/
β”‚ β”‚ β”œβ”€β”€ ControlPanel.tsx # Settings and history UI
+β”‚ β”‚ β”œβ”€β”€ OnboardingFlow.tsx # First-time setup wizard
+β”‚ β”‚ β”œβ”€β”€ SettingsPage.tsx # Settings interface
β”‚ β”‚ β”œβ”€β”€ ui/ # shadcn/ui components
β”‚ β”‚ β”‚ β”œβ”€β”€ button.tsx
β”‚ β”‚ β”‚ β”œβ”€β”€ card.tsx
@@ -117,6 +141,10 @@ open-wispr/
β”‚ β”‚ β”‚ └── tooltip.tsx
β”‚ β”‚ └── lib/
β”‚ β”‚ └── utils.ts # Utility functions
+β”‚ β”œβ”€β”€ services/
+β”‚ β”‚ └── ReasoningService.js # AI processing and agent name detection
+β”‚ β”œβ”€β”€ utils/
+β”‚ β”‚ └── agentName.ts # Agent name management utility
β”‚ └── components.json # shadcn/ui configuration
└── assets/ # App icons and resources
```
diff --git a/src/components/OnboardingFlow.tsx b/src/components/OnboardingFlow.tsx
index 2b23adff8..7fbfce469 100644
--- a/src/components/OnboardingFlow.tsx
+++ b/src/components/OnboardingFlow.tsx
@@ -17,6 +17,7 @@ import {
Sparkles,
Lock,
X,
+ User,
} from "lucide-react";
import TitleBar from "./TitleBar";
import WhisperModelPicker from "./WhisperModelPicker";
@@ -33,6 +34,7 @@ import { useClipboard } from "../hooks/useClipboard";
import { useSettings } from "../hooks/useSettings";
import { getLanguageLabel, getReasoningModelLabel } from "../utils/languages";
import LanguageSelector from "./ui/LanguageSelector";
+import { setAgentName as saveAgentName } from "../utils/agentName";
interface OnboardingFlowProps {
onComplete: () => void;
@@ -68,6 +70,7 @@ export default function OnboardingFlow({ onComplete }: OnboardingFlowProps) {
const [apiKey, setApiKey] = useState(openaiApiKey);
const [hotkey, setHotkey] = useState(dictationKey || "`");
+ const [agentName, setAgentName] = useState("Agent");
const { alertDialog, showAlertDialog, hideAlertDialog } = useDialogs();
const practiceTextareaRef = useRef<HTMLInputElement>(null);
@@ -82,6 +85,7 @@ export default function OnboardingFlow({ onComplete }: OnboardingFlowProps) {
{ title: "Permissions", icon: Shield },
{ title: "Hotkey", icon: Keyboard },
{ title: "Test", icon: TestTube },
+ { title: "Agent Name", icon: User },
{ title: "Finish", icon: Check },
];
@@ -104,6 +108,7 @@ export default function OnboardingFlow({ onComplete }: OnboardingFlowProps) {
const saveSettings = useCallback(async () => {
updateTranscriptionSettings({ whisperModel, preferredLanguage });
setDictationKey(hotkey);
+ saveAgentName(agentName);
localStorage.setItem(
"micPermissionGranted",
@@ -123,6 +128,7 @@ export default function OnboardingFlow({ onComplete }: OnboardingFlowProps) {
whisperModel,
hotkey,
preferredLanguage,
+ agentName,
permissionsHook.micPermissionGranted,
permissionsHook.accessibilityPermissionGranted,
useLocalWhisper,
@@ -597,7 +603,53 @@ export default function OnboardingFlow({ onComplete }: OnboardingFlowProps) {
</div>
);
- case 6: // Complete
+ case 6: // Agent Name
+ return (
+ <div className="space-y-6">
+ <div className="text-center">
+ <h2 className="text-2xl font-bold text-stone-900 mb-2">
+ Name Your Agent
+ </h2>
+ <p className="text-stone-600">
+ Give your agent a name so you can address it specifically when
+ giving instructions.
+ </p>
+ </div>
+
+ <div className="space-y-4 p-4 bg-gradient-to-r from-purple-50 to-blue-50 border border-purple-200 rounded-xl">
+ <h4 className="font-medium text-purple-900 mb-3">
+ πŸ’‘ How this helps:
+ </h4>
+ <ul className="text-sm text-purple-800 space-y-1">
+ <li>
+ β€’ Say "Hey {agentName || "Agent"}, write a formal email" for
+ specific instructions
+ </li>
+ <li>
+ β€’ Use the name to distinguish between dictation and commands
+ </li>
+ <li>β€’ Makes interactions feel more natural and personal</li>
+ </ul>
+ </div>
+
+ <div className="space-y-4">
+ <label className="block text-sm font-medium text-gray-700 mb-2">
+ Agent Name
+ </label>
+ <Input
+ placeholder="e.g., Assistant, Jarvis, Alex..."
+ value={agentName}
+ onChange={(e) => setAgentName(e.target.value)}
+ className="text-center text-lg font-mono"
+ />
+ <p className="text-xs text-gray-500 mt-2">
+ You can change this anytime in settings
+ </p>
+ </div>
+ </div>
+ );
+
+ case 7: // Complete
return (
<div className="text-center space-y-6">
<div className="w-16 h-16 mx-auto bg-green-100 rounded-full flex items-center justify-center">
@@ -637,6 +689,10 @@ export default function OnboardingFlow({ onComplete }: OnboardingFlowProps) {
{getLanguageLabel(preferredLanguage)}
</span>
</div>
+ <div className="flex justify-between">
+ <span>Agent Name:</span>
+ <span className="font-medium">{agentName}</span>
+ </div>
<div className="flex justify-between">
<span>Permissions:</span>
<span className="font-medium text-green-600">
@@ -685,6 +741,8 @@ export default function OnboardingFlow({ onComplete }: OnboardingFlowProps) {
case 5:
return true; // Practice step is always ready to proceed
case 6:
+ return agentName.trim() !== ""; // Agent name step
+ case 7:
return true;
default:
return false;
diff --git a/src/components/SettingsPage.tsx b/src/components/SettingsPage.tsx
index 2ab6d645e..aa1a56d34 100644
--- a/src/components/SettingsPage.tsx
+++ b/src/components/SettingsPage.tsx
@@ -13,6 +13,7 @@ import {
Brain,
Shield,
ChevronRight,
+ User,
} from "lucide-react";
import TitleBar from "./TitleBar";
import WhisperModelPicker from "./WhisperModelPicker";
@@ -21,6 +22,7 @@ import ApiKeyInput from "./ui/ApiKeyInput";
import { ConfirmDialog, AlertDialog } from "./ui/dialog";
import { useSettings } from "../hooks/useSettings";
import { useDialogs } from "../hooks/useDialogs";
+import { useAgentName } from "../utils/agentName";
import { useWhisper } from "../hooks/useWhisper";
import { usePermissions } from "../hooks/usePermissions";
import { useClipboard } from "../hooks/useClipboard";
@@ -35,6 +37,7 @@ type SettingsSectionType =
| "transcription"
| "reasoning"
| "hotkey"
+ | "agentName"
| "updates"
| "about";
@@ -101,6 +104,7 @@ export default function SettingsPage({ onBack }: SettingsPageProps) {
const whisperHook = useWhisper(showAlertDialog);
const permissionsHook = usePermissions(showAlertDialog);
const { pasteFromClipboardWithFallback } = useClipboard(showAlertDialog);
+ const { agentName, setAgentName } = useAgentName();
useEffect(() => {
whisperHook.checkWhisperInstallation();
@@ -270,6 +274,11 @@ export default function SettingsPage({ onBack }: SettingsPageProps) {
label: "Hotkey Setup",
icon: Keyboard,
},
+ {
+ id: "agentName" as SettingsSectionType,
+ label: "Agent Name",
+ icon: User,
+ },
{
id: "updates" as SettingsSectionType,
label: "App Updates",
@@ -556,6 +565,93 @@ export default function SettingsPage({ onBack }: SettingsPageProps) {
</div>
);
+ case "agentName":
+ return (
+ <div className="space-y-6">
+ <div>
+ <h3 className="text-lg font-semibold text-gray-900 mb-2">
+ Agent Name Configuration
+ </h3>
+ <p className="text-sm text-gray-600 mb-6">
+ Customize your agent's name to make interactions feel more
+ personal and natural.
+ </p>
+ </div>
+
+ <div className="space-y-4 p-4 bg-gradient-to-r from-purple-50 to-blue-50 border border-purple-200 rounded-xl">
+ <h4 className="font-medium text-purple-900 mb-3">
+ πŸ’‘ How to use agent names:
+ </h4>
+ <ul className="text-sm text-purple-800 space-y-2">
+ <li>
+ β€’ Say "Hey {agentName}, write a formal email" for specific
+ instructions
+ </li>
+ <li>
+ β€’ Use "Hey {agentName}, format this as a list" for text
+ enhancement commands
+ </li>
+ <li>
+ β€’ The agent will recognize when you're addressing it directly
+ vs. dictating content
+ </li>
+ <li>
+ β€’ Makes conversations feel more natural and helps distinguish
+ commands from dictation
+ </li>
+ </ul>
+ </div>
+
+ <div className="space-y-4 p-4 bg-gray-50 border border-gray-200 rounded-xl">
+ <h4 className="font-medium text-gray-900">Current Agent Name</h4>
+ <div className="flex gap-3">
+ <Input
+ placeholder="e.g., Assistant, Jarvis, Alex..."
+ value={agentName}
+ onChange={(e) => setAgentName(e.target.value)}
+ className="flex-1 text-center text-lg font-mono"
+ />
+ <Button
+ onClick={() => {
+ setAgentName(agentName.trim());
+ showAlertDialog({
+ title: "Agent Name Updated",
+ description: `Your agent is now named "${agentName.trim()}". You can address it by saying "Hey ${agentName.trim()}" followed by your instructions.`,
+ });
+ }}
+ disabled={!agentName.trim()}
+ >
+ Save
+ </Button>
+ </div>
+ <p className="text-xs text-gray-600 mt-2">
+ Choose a name that feels natural to say and remember
+ </p>
+ </div>
+
+ <div className="bg-blue-50 p-4 rounded-lg">
+ <h4 className="font-medium text-blue-900 mb-2">
+ 🎯 Example Usage:
+ </h4>
+ <div className="text-sm text-blue-800 space-y-1">
+ <p>
+ β€’ "Hey {agentName}, write an email to my team about the
+ meeting"
+ </p>
+ <p>
+ β€’ "Hey {agentName}, make this more professional" (after
+ dictating text)
+ </p>
+ <p>β€’ "Hey {agentName}, convert this to bullet points"</p>
+ <p>
+ β€’ Regular dictation: "This is just normal text" (no agent name
+ needed)
+ </p>
+ </div>
+ </div>
+ </div>
+ );
+
case "updates":
return (
<div className="space-y-6">
diff --git a/src/services/ReasoningService.js b/src/services/ReasoningService.js
index 506219aa0..045b7c443 100644
--- a/src/services/ReasoningService.js
+++ b/src/services/ReasoningService.js
@@ -1,6 +1,33 @@
import { getModelProvider } from "../utils/languages.ts";
+import { getAgentName } from "../utils/agentName.ts";
+
+const getReasoningPrompt = (text, agentName) => {
+ const hasAgentReference =
+ agentName &&
+ (text.toLowerCase().includes(`hey ${agentName.toLowerCase()}`) ||
+ text.toLowerCase().includes(agentName.toLowerCase()));
+
+ if (hasAgentReference) {
+ return `You are ${agentName}, an AI text formatting assistant. The user has addressed you by name and is giving you specific instructions to process their text.
+
+Your role:
+1. When addressed as "${agentName}" or "Hey ${agentName}", you are being given instructions about how to format or process text
+2. Look for commands like:
+ - "${agentName}, make this more professional"
+ - "${agentName}, format this as a list"
+ - "${agentName}, write an email about..."
+ - "${agentName}, convert this to bullet points"
+3. Follow the user's formatting instructions while preserving their intent
+4. If asked to write content, create it based on their request
+5. Remove the agent name references from the final output (don't include "Hey ${agentName}" in your response)
+6. For editing commands, apply the requested changes to the text that follows
+
+Transcript with instructions:
+"{{text}}"
-const REASONING_PROMPT = `You are a text formatting assistant. Your job is to clean up and format voice-to-text transcriptions while preserving the speaker's natural tone and intent.
+Processed text:`;
+ } else {
+ return `You are a text formatting assistant. Your job is to clean up and format voice-to-text transcriptions while preserving the speaker's natural tone and intent.
Rules:
1. If the speaker gives instructions like "scratch that", "ignore that", "delete the previous part", "never mind", or similar - follow them and remove the referenced content
@@ -14,6 +41,8 @@ Transcript to format:
"{{text}}"
Formatted text:`;
+ }
+};
class ReasoningService {
constructor() {
@@ -40,7 +69,11 @@ class ReasoningService {
async processWithOpenAI(text, model) {
const apiKey = await this.getAPIKey("openai");
- const prompt = REASONING_PROMPT.replace("{{text}}", text);
+ const agentName = getAgentName();
+ const prompt = getReasoningPrompt(text, agentName).replace(
+ "{{text}}",
+ text
+ );
const maxTokens = Math.max(100, text.length * 2);
const response = await fetch("https://api.openai.com/v1/chat/completions", {
@@ -74,7 +107,11 @@ class ReasoningService {
async processWithAnthropic(text, model) {
const apiKey = await this.getAPIKey("anthropic");
- const prompt = REASONING_PROMPT.replace("{{text}}", text);
+ const agentName = getAgentName();
+ const prompt = getReasoningPrompt(text, agentName).replace(
+ "{{text}}",
+ text
+ );
const maxTokens = Math.max(100, text.length * 2);
const response = await fetch("https://api.anthropic.com/v1/messages", {
diff --git a/src/utils/agentName.ts b/src/utils/agentName.ts
new file mode 100644
index 000000000..dc9eaeb9d
--- /dev/null
+++ b/src/utils/agentName.ts
@@ -0,0 +1,26 @@
+import { useState } from "react";
+
+const AGENT_NAME_KEY = "agentName";
+
+export const getAgentName = (): string => {
+ return localStorage.getItem(AGENT_NAME_KEY) || "Agent";
+};
+
+export const setAgentName = (name: string): void => {
+ localStorage.setItem(AGENT_NAME_KEY, name);
+};
+
+export const clearAgentName = (): void => {
+ localStorage.removeItem(AGENT_NAME_KEY);
+};
+
+export const useAgentName = () => {
+ const [agentName, setAgentNameState] = useState<string>(getAgentName());
+
+ const updateAgentName = (name: string) => {
+ setAgentName(name);
+ setAgentNameState(name);
+ };
+
+ return { agentName, setAgentName: updateAgentName };
+};